---
title: 'Enhancement and Reusage of Biomedical Knowledge Graph Subsets'
title_short: 'Enhancement and reuse of Knowledge Grpahs'
tags:
- Wikidata
- ShEx
- RDF
- Knowledge graphs
authors:
- name: Jose Emilio Labra Gayo
orcid: 0000-0001-8907-5348
affiliation: 1
- name: Ammar Ammar
affiliation: 2
- name: Dan Brickley
affiliation: 3
- name: Eric Prud'hommeaux
orcid: 0000-0003-1775-9921
affiliation: 5
- name: Seyed Amir Hosseini Beghaeiraveri
orcid: 0000-0002-9123-5686
affiliation: 4
- name: Carolina González-Cavazos
affiliation: 8
- name: Sabah Ul-Hasan
affiliation: 8
- name: Andra Waagmeester
affiliation: 7
- name: Nils Hoffmann
affiliation: 9

affiliations:
- name: WESO research group, University of Oviedo, Spain
index: 1
- name: Maastricht University
index: 2
- name: Google, London, UK
index: 3
- name: Heriot-Watt University, UK
index: 4
- name: Janeiro Digital, W3C/MIT
index: 5
- name: Micelio/Gene Wiki
index: 7
- name: Gene Wiki Project, Scripps Research Institute
index: 8
- name: Nils affiliation
index: 9

date: 11 November 2022
bibliography: paper.bib
event: BH22EU
group: Enhancement and Reusage of Biomedical Knowledge Graph Subsets
authors_short: Jose E. Labra \emph{et al.}
---
# Enhancement and Reusage of Biomedical Knowledge Graph Subsets
## Abstract

Knowledge Graphs (KGs) such as Wikidata act as a hub of information from multiple domains and disciplines, and is crowdsourced by multiple stakeholders. The vast amount of available information makes it difficult for researchers to manage the entire KG, which is also continually being edited and changing its content. It is necessary to develop tools that extract snapshots and subsets for some specific domains of interest. These subsets help researchers by reducing costs and ease accessbility to data of interest. In the last two biohackathons, we have identified this issue and created prototypes to extract subsets easily applicable to Wikidata, as well as to define a map of the different approaches used to tackle this problem. Building on those outcomes, we aim to enhance subsetting in both definitions using Entity schemas based on Shape Expressions and extraction algorithms, with a special focus on the biomedical domain captured by entity schemas like the one defined in the GeneWiki project. Our first aim is to develop complex subsetting patterns to cover subsetting based on qualifiers and references for enhancing credibility of datasets. Our second aim is to establish a faster subsetting extraction platform applying new algorithms based on Apache Spark and new tools like a document-oriented DBMS platform. During this biohackathon, we aim to explore reuse workflows of Wikidata subsets specifically with respect to drug repurposing. The biohackathon will assist in an evaluation of existing nodes and edges on drug-target interactions categories within Wikidata, and if these are in need of updates as well as deeper annotation. We would also aim to deliver machine readable schemas of drug-target interactions in Wikidata for future data reuse.

## Report about project 11 at Biohackathon-Europe 2022

## Links

- [HackMD notes](https://hackmd.io/@vfttImB6SdiREhh65hglIA/SykVQELBi/edit)
- [notes from 2021 (Barcelona)](https://hackmd.io/gHHasGifTEaDMdvM9LTxTQ)

# Fourth day 10th-Nov-2022

## Attendants

- Seyed
- Carolina
- Ammar
- Labra

## Notes
- Labra
- Published dataset about GeneWiki in Zenodo: https://doi.org/10.5281/zenodo.7309739
- Waiting for the other subsets that are being generated: more than 20hours...don't know why
- Uploaded to a local Fuseki SPARQL endpoint the GeneWiki dataset and seems to work:

PREFIX ps: <https://w3id.org/payswarm#>
prefix geo:      <http://www.opengis.net/ont/geosparql#> 
prefix p:        <http://www.wikidata.org/prop/> 
prefix pq:       <http://www.wikidata.org/prop/qualifier/> 
prefix pr:       <http://www.wikidata.org/prop/reference/> 
prefix prov:     <http://www.w3.org/ns/prov#> 
prefix ps:       <http://www.wikidata.org/prop/statement/> 
prefix rdf:      <http://www.w3.org/1999/02/22-rdf-syntax-ns#> 
prefix rdfs:     <http://www.w3.org/2000/01/rdf-schema#> 
prefix schema:   <http://schema.org/> 
prefix skos:     <http://www.w3.org/2004/02/skos/core#> 
prefix wd:       <http://www.wikidata.org/entity/> 
prefix wdno:     <http://www.wikidata.org/prop/novalue/> 
prefix wds:      <http://www.wikidata.org/entity/statement/> 
prefix wdt:      <http://www.wikidata.org/prop/direct/> 
prefix wikibase: <http://wikiba.se/ontology#> 
prefix xsd:      <http://www.w3.org/2001/XMLSchema#> 


SELECT * WHERE {
  ?cellular_component wdt:P702 ?gene .
  OPTIONAL { ?cellular_component rdfs:label ?ccLabel . }
  OPTIONAL { ?gene rdfs:label ?geneLabel . }
 filter(
    lang(?ccLabel)='en' 
    # && lang(?ccLabel)='es'
  )
} 

It obtained 365 results

- Ammar
- Created a subset with lipids using the WholeEntity and this [Shape Expression](https://github.com/kg-subsetting/biohackathon2022/blob/main/workflows/wdsub-lipids/lipids.shex)
- 1.6Gb gzipped, 8.4Gb unzipped
- Trying to guess if the problem is the OR in the ShEx that seems to ignore the first part...maybe to version 0.0.29 of wdsub could solve it.
- We are going to create a new subset using this simplified ShEx
- Ammar will help publishing the Zenodo dataset in Turtle in a public SPARQL endpoint with a given IP so we can do SPARQL queries on it...

- Seyed
- Script to count number of instance between WDF and WDSub but it didn't work
- Trying to check which items were extraced by one and not the other...
- One was created with C and the other with Blazegraph
- 600 items in wdf that are not in wdsub
- Carolina will try to help there
- Carolina
- Waiting for the JSON results to analyze them using some link prediction algorithms...
- Started to look at processing turtle rdflib
- Will take a look at the differences reported by Seyed and the Python code
- RDF2vec library could be used for this: http://rdf2vec.org/
- Q. How is the embedding happening?
- Random walks...random visits of the nodes...
- Train neural network with those connections...
- Principal components labelled with the class...
- One issue could be to rewrite from wdt:P31 to rdf:type
- Fact validation with KG embeddings
- https://ceur-ws.org/Vol-2456/paper33.pdf

- Semantic Answer Type prediction (ISWC 2020-2021)
- http://ceur-ws.org/Vol-2774/paper-05.pdf
- http://ceur-ws.org/Vol-3119/paper7.pdf

## Meeting at 16h on 10th nov 2022

Ammar took the GeneWiki subset previously generated and published it a Virtuoso TripleStore with a public SPARQL endpoint. After that, we were able to create several SPARQL queries to analyse the extracted subset.

- Some example queries are the following:

- We were discussing about the publication of the results of the biohackathon and decided the following:
- Publish biohackarxiv report of last year's which we had left as a draft unpublished in the next weeks
- Try to publish a first draft of the biohackarxiv report tomorrow.

- We started a discussion about possible reproducibility pipelines
- Embed docker invocation in a Jupyter notebook. This option requires infrastructure to host the Jupyter notebooks.
- Embed a Jupyter notebook in a docker container. This is possible using [DataScience Notebook](https://hub.docker.com/r/jupyter/datascience-notebook).
- Another possibility could be to use a Workflow language like [SnakeMake](https://snakemake.readthedocs.io/en/stable/).

- Next steps:
- Have the subsets online and update them periodically. Ammar will consult the availability of resources.
- Requirements and challenges to consider
- Updating the shapes.
- Regularly downloading dumps, something like a cron script and updating the subset.
- Save the shape expression in a github repo and take the latest version from there.
- Trying to implement a: Wikidata subsetting as a Service
- 2 possibilities:
- Curated list of Shape Expressions/subsets that are regularly generated for specific purposes.
- A service like WDumper that takes as input a Shape Expression
- Types of analysis that we can perform on the subsets:
- Link prediction/classification. Drug prediction
- Analyzing data quality: references and qualifiers...subsets can be considered as samples of wikidata to investigate the quality of the data.
- Future use cases:
- Subsets as starting points as a seed to create new knowledge graphs, integrating their information with other sources. One example, could be mapping with data from other ontologies, KGs, etc. Another use case is bioschemas.
- Visualization of subsets which can be specific for some domain.
- Use cases about using subgraphs for machine learning/deep learning/question answering algorithms...help NLP algorithms.

# Third day 9th-Nov-2022

## Attendants
- Seyed
- Jose Labra
- Andra
- Ammar
- Carolina

## Notes

- Explanation about wdsub
- Input JSON dumps
- Output: you can choose between JSON or Turtle
- The dataset generated yesterday from Genewiki seems to generate only taxons...
- Maybe the reason is that the JSOn output generation has some bug
- Ammar used the ShEx for the lipids with wdsub library and used it with the 2014...it took less than an hour
- 64M zipped, 492 Mb unzipped
- Only contains the taxon entities...maybe the same bug?
- Upload the dataset of lipids to another Zenodo
- Andra:
- wdsub seems to run
- shape expressions with classes and identifiers...
- error messages
- somevalues/novalues are not implemented yet
- geocoordinates which should not be in genewiki
- took a long time:
- 14 hours more or less
- 610Mb
- Detect problems in wikidata
- taxon is not a taxon but a taxon name
- multiple wikidata items for the same taxon
- problem with bacteria and viruses
- targeted subsets
- taxon subset...
- Carolina:
- GeneWiki subset but seems to contain only taxons
- Github: https://github.com/kg-subsetting/biohackathon2022/blob/main/examples/ShEx/geneWiki.shex
- Apply the subset to machine learning algorithm that she has (Case-Based reasoning).
- Apply her algorithm to this subset for drug repurpusing.
- Seyed:
- Checking the number of extracted outputs
- WDF: Wikibase dump filter
- Dumpformat = JSON
- both tools miss some gene instances ~ 4000 instances
- wdsub miss 600 more
- List of items that are in one and not in the other
- Analyzing accuracy, which is important
- Labra
- Created the GeneWiki subset using the GeneWiki.shex using JSON output for 2018
- Create the script to obtain
- GeneWiki of 2018 in Turtle
- GeneWiki of 2022 in Turtle
- GeneWiki of 2022 in Json
- Still running (more than 12 hours)
- Publish in Zenodeo
- Possibilities to upload the datasets
- github repo for datasets: https://github.com/kg-subsetting/datasets-biohackathon2022
- Zenodo: already done, limit of 50G
- Dryad: maybe for the latest version for the paper, because the datasets are reviewed
- data.world
- Examples from book: https://data.world/swwo
- tryplydb ?
- Discuss the workflow and automate the creation of subset
- Wikidata Subsets as as Service
- Enter a entity schema and obtain a subset
- wdsub is a command line tool with docker support
- Andra was able to do it in a laptop (slimbook)
- We could wrap it into a web service
- It would be great to have a server host as WDumper
- https://wdumps.toolforge.org/dumps
- Options
- Command line
- Generic web service
- Pre-defined subsets
- Fix set of curated shape expressions
- cron job to run that service...
- Wikidata + quality control
- Other tools apart from wdsub
- Ammar reminds that he created a subset for lipids using pyshex and slurper

# Second day 8th-Nov-2022

## Attendants
- Seyed
- Jose Labra
- Andra

## Notes
- Seyed, created a subset which contains instances of Genes, Proteins, etc. and put the output (turtle) in a Blazegraph
- Dumps from: 3 January 2022 [Link]( https://academictorrents.com/details/229cfeb2331ad43d4706efd435f6d78f40a3c438)
- SPARQL counting:
- Gene: 1196517
- Protein: 987614
- Chemical compound: 1244866
- Disease: 5512
- Andra [created an example](https://github.com/kg-subsetting/biohackathon2022/blob/main/examples/ShEx/gene.shex) of a ShEx to extract info related with Genes but that is more inclusive than just checking the P31 Q7187, including all things that have at least some specific properties...

<Gene> @<TypedGene> OR @<PropertyBasedGene>

<TypedGene> EXTRA p:P31 {
  p:P31 { ps:P31 [wd:Q7187]} 
}

<PropertyBasedGene> 
  { p:P351 { ps:P351 . } +  OR 
  { p:P353 { ps:P353 . } + OR 
  . . .
}

- Ammar is planning to work on a subset related with lipids using a ShEx like the following:

PREFIX xsd:    <http://www.w3.org/2001/XMLSchema#>
PREFIX :    <http://bigcat.unimaas.nl/>
PREFIX rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs:    <http://www.w3.org/2000/01/rdf-schema#>
PREFIX wd:    <http://www.wikidata.org/entity/>
PREFIX wdt:    <http://www.wikidata.org/prop/direct/>

start= @:lipids OR @:taxon

:lipids {
   wdt:P2063 .+;
   wdt:P234 .+;
   wdt:P235 .+;
   wdt:P703 @:taxon +;
}

:taxon {
	wdt:P31 [wd:Q16521];
}

#!/bin/sh
Version=0.0.26
Dump2018=wikidata-20180115-all # Local Downloaded dump from Wikidata

Name=taxons
DumpFile=${Dump2018}
SchemaFormat=ShExC
DumpMode=OnlyMatched
DumpFormat=Turtle
docker run -d -v /home/labra/dumps:/data -v /home/labra/wdsubConfig/shex:/shex -v /home/labra/generatedDumps:/generatedDumps wesogroup/wdsub:${Version} dump -o /generatedDumps/${Name}.ttl.gz -s /shex/${Name}.shex /data/${DumpFile}.json.gz --dumpFormat ${DumpFormat} --dumpMode ${DumpMode} --schemaFormat ${SchemaFormat}

Ideas talking with Carolina during lunch
- Create a subset based on her [ShEx](https://github.com/kg-subsetting/biohackathon2022/blob/main/examples/ShEx/geneWiki.shex)
- Review JSON structure of the subsets
- Once we get the subset, apply some Machine Learning algorithms...
- Link prediction?
- We started to run wdsub using Carolina's ShEx which contains information about GeneWiki. The shape we used was [this one](https://github.com/kg-subsetting/biohackathon2022/blob/main/examples/ShEx/geneWiki.shex)
- [Link to some past results from Labra's students]( https://docs.google.com/document/d/1mxEo6y4IJjVpDK1nT2PvcvkFgvTCLBJax6WipTMXCUM/edit#heading=h.9ywt0nihccfc)
-

# First day 7th-Nov-2022
## Attendants
- Jose Labra
- Andra Waagmeester
- Eric Prud'hommeaux
- Carolina Gonzalez
- Sayed Beghaei

## Notes

- During the break I met Egon and were talking about possibility of creating a subset about Chemistry
- Idea: prepare a tutorial to explain how to create subsets using wdsub
- Andra: plan for the week
- Expectations
- Finish the paper that we started last year
- Create subsets for GeneWiki
- What to do with the subset
- Issue to solve: Where to store the created subset
- [Dryad](https://datadryad.org/stash)
- Seyed same expectations
- Expressivity of wdsub is constrained by sequential processing for each line...it checks each item

# Gameplan
* Andra: Revisit the wdsubset of genewiki subgraph
* This involves support for qualifiers and references

Example of things supported in wdsub:

start = @<Gene> OR @<Disease> OR ...

<Gene> EXTRA wdt:P31 {
  wdt:P31   [ wd:Q7187 ....] ;
  # Don't follow this pattern: wdt:P31 . * ;
  wdt:P703  
  
  # References are supported but not validated
  :encodes  @<Protein> ;
  
  # Qualifiers...
  p:P166 { 
    ps:P166  @<Award>         ;
    pq:P585  xsd:dateTime   ? ; 
    pq:P1706 @<Researcher>  *
  } *

}



<Disiease> {

}

<Protein> { 
  wdt:p31 [....]
}

start = @<Gene_by_ontology> OR @<Gene_by_identifier>

<Gene_by_ontology> {
    p:P31 {
       ps:P31 [wd:Q7187] ;
       prov:wasDerivedFrom . ;
    }
    
    p:P703 {
        ps:P703 [wd:homo sapiens] ;
        prov:wasDerivedFrom . ;
     
    }
}

<Gene_by_identifier> {
    p:
}

Seyed's question: What is the difference between these (in terms of extracting subset via WDSub)?:
1)

start = @<Gene> OR @<Disease> OR <Protein>

<Gene> EXTRA wdt:P31 {
  wdt:P31   [ wd:Q7187] ;
}

<Disiease> {
    wdt:p31 [wd:Qabc]
}

<Protein> { 
  wdt:p31 [wd:Qijk]
}

and 2)

start = @<all>

<all> EXTRA wdt:P31 {
  wdt:P31   [ wd:Q7187, wd:Qabc, wd:Qijk] ;
}

- Meeting with virtual attendees, tomorrow at 10h CET

https://github.com/kg-subsetting/biohackathon2022

- Andra: will work on preparing some examples about GeneWiki
- Seyed: Check if the tool finnished and review the output of wdsub.
- Report about the possible turtle output
- Carolina has some Shape Expressions about GeneWiki (add more properties)
- Labra will continue checking the tool and update docker image

run dump -s schema.shex --schemaFormat ShExC -o target/exampleReferences.ttl.gz input.json.gz --dumpMode OnlyMatched --dumpFormat Turtle