# 圖資料庫與neo4j基本介紹 [江豪文](https://howard-haowen.rohan.tw/) ---- ## 大綱 - 圖資料庫簡介 - Neo4j課程與證照 - Neo4j生態系操作介面介紹 - Neo4j Desktop - Neo4j Bloom - Neo4j Graph Data Science Library - Neo4j Browser - Cypher語法 - 讀取DB - 寫入DB --- ## 圖資料庫簡介 - 圖資料庫是近年來受歡迎程度成長最快的DB類型  ---- ## 圖資料庫的主要優勢 - 儲存複雜的網絡關係 - 高效能、高擴展 - 密切結合圖演算法與AI應用 - 友善的查詢語言 ---- - 儲存複雜的網絡關係  ---- - 高效能、高擴展 :chart_with_upwards_trend: 關聯式資料庫與圖資料庫效能比較 :dart: 一百萬筆客戶資料，每位客戶有50位親友  [資料來源](https://neo4j.com/blog/why-graph-data-relationships-matter/) ---- - 密切結合圖演算法與AI應用  ---- - 以圖資料庫為後端的應用架構 - REST API  ---- - 以圖資料庫為後端的應用架構 - GraphQL  ---- ## 常見的圖資料庫管理系統  --- ## Neo4j課程與證照 - 資應課規劃課程 - 本次分享會 - 由Neo4j官方師資群教授四堂課程（每堂八小時）、兩場Workshop - 官方免費線上課程 - [GraphAcademy](https://graphacademy.neo4j.com/) - [官方認可證照](https://graphacademy.neo4j.com/certification/) - Neo4j Certified Professional - Neo4j Graph Data Science Certification ---- ## 圖資料庫的基本組成 - 節點 - 關係  ---- ## 命名慣例 - 節點 `PolicyHolder` - 關係 `MARRIED_TO` - 屬性 `policyNumber`  ---- ## 金融場景範例  [圖片來源](https://www.tigergraph.com/blog/how-the-worlds-largest-banks-use-advanced-graph-analytics-to-fight-fraud/) ---- - 傳統防詐預測分析透過尋找偏差值，而圖資料庫有利透過查詢語法或圖演算法偵測資料常規值內的詐騙集團  [圖片來源](https://www.larus-ba.it/2020/04/16/insurance-fraud-detection-with-neo4j/) --- ## Neo4j生態系操作介面介紹 - Neo4j Desktop - Neo4j Bloom - Neo4j Graph Data Science Library - Neo4j Browser ---- ## Neo4j Desktop  ---- - DBMS vs DB  ---- ## Neo4j Bloom  ---- ## Neo4j Graph Data Science Library  ---- - 何謂Graph Data Science  ---- - 圖演算法的類別  ---- - 以原生圖為核心的AI流程  ---- - 原生圖計算與下一代AI進程  ---- ## Neo4j Browser  ---- - 顯示當前DBMS內的所有DB ```sql! SHOW DATABASES ``` ```sql! :dbs ``` | name | access| address | | -------- | -------- | -------- | | "system" | "read-write" | "localhost:11003" | | "neo4j" | "read-write" | "localhost:11003" | | "creditcard" | "read-write" | "localhost:11003" | | "bankfraud" | "read-write" | "localhost:11003" | ---- - 切換當前DB ```sql! :USE neo4j ``` - 顯示當前DB結構綜覽 ```sql! CALL db.schema.visualization ```  ---- - 顯示當前DB內所有節點的標籤 ```sql! CALL db.labels() ``` | "label" | | -------- | | "Movie" | | "Person" | ---- - 顯示當前DB內所有關係的類別 ```sql! CALL db.relationshipTypes() ``` | "relationshipType" | | -------- | | "ACTED_IN" | | "DIRECTED" | | "PRODUCED" | | "WROTE" | | "FOLLOWS" | | "REVIEWED" | --- ## Cypher語法 :+1:[精簡版語法查找表](https://neo4j.com/docs/cypher-cheat-sheet/current/) :100: [完整版語法參考書](https://s3.amazonaws.com/artifacts.opencypher.org/openCypher9.pdf) :sparkles: [語法風格指引](https://s3.amazonaws.com/artifacts.opencypher.org/M21/docs/style-guide.pdf) - 讀取DB - 寫入DB ---- ## 讀取DB - `MATCH` {pattern}: 根據模式尋找 - `WHERE` {condition}: 以條件過濾結果 - `RETURN`{result}: 返回結果 --- ## 尋找節點`()` - 全部節點 ```sql! MATCH (node) RETURN node ```  ---- ## 以標籤篩選節點 - 節點標籤為`Customer` ```sql! MATCH (c:Customer) RETURN c ``` ```graphviz digraph hierarchy { nodesep=1.0 客戶A -> 客戶B -> 客戶C 客戶A -> 客戶C } ``` ---- ## 任務:question: - 找出標籤為`Person`的節點 ```sql! MATCH () RETURN ``` ---- ## 任務:100: - 找出標籤為`Person`的節點 ```sql! MATCH (p:Person) RETURN p ``` ---- ## 以屬性過濾節點 - 提供屬性辭典 ```sql! MATCH (c:Customer {givenName: '小新'}) RETURN c ``` ```graphviz digraph hierarchy { nodesep=1.0 張小新 -> 陳小新 張小新 -> 林小新 } ``` ---- ## 以屬性過濾節點 - 提供`WHERE`子句 ```sql! MATCH (c:Customer) WHERE c.givenName = '小新' RETURN c ``` ```graphviz digraph hierarchy { nodesep=1.0 張小新 -> 陳小新 張小新 -> 林小新 } ``` ---- ## 任務:question: - 找出標籤為`Person`並且在1980年(含)以後出生(屬性名稱`born`)的節點 ```sql! MATCH () RETURN ``` ---- ## 任務:100: - 找出標籤為`Person`並且在1980年(含)以後出生(屬性名稱`born`)的節點 ```sql! MATCH (p:Person) WHERE p.born >= 1980 RETURN p ``` --- ## 尋找關係 - 節點之間有單一方向的關係`-->` ```sql! MATCH (p:Person)-->(c:Car) RETURN p, c ```  ---- ## 以類別篩選關係 - 要保人與受益人的關係類別為`IS_PARENT` ```sql! MATCH (p:PolicyHolder)-[r:IS_PARENT]->(b:Beneficiary) RETURN p, b ``` ```graphviz digraph hierarchy { nodesep=1.0 林大新 -> 林小新[label="父母關係",weight="0.2"] 張大光 -> 陳小新[label="父母關係",weight="0.2"] 林大新 -> 張大光[label="父母關係",weight="0.2"] } ``` ---- ## 以屬性過濾關係 - 要保人與受益人的關係類別為`IS_RELATED` ```sql! MATCH (p:PolicyHolder)-[r:IS_RELATED]->(b:Beneficiary) WHERE r.code = 2 RETURN p, b ``` ```graphviz digraph hierarchy { nodesep=1.0 林大新 -> 林小新[label="父母關係",weight="0.2"] 張大光 -> 陳小新[label="父母關係",weight="0.2"] 林大新 -> 張大光[label="父母關係",weight="0.2"] } ``` ---- ## 任務:question: - 找出`Person`和`Movie`節點之間有`DIRECTED`關係的所有節點 ```sql! MATCH ()-[]->() RETURN ``` ---- ## 任務:100: - 找出`Person`和`Movie`節點之間有`DIRECTED`關係的所有節點 ```sql! MATCH (p:Person)-[r:DIRECTED]->(m:Movie) RETURN p, m ``` ---- ## 以多個類別篩選關係 - 要保人與受益人的關係標籤為`IS_PARENT`或`IS_SPOUSE` ```sql! MATCH (p:PolicyHolder)-[r:IS_PARENT|IS_SPOUSE]->(b:Beneficiary) RETURN p, b ``` ```graphviz digraph hierarchy { nodesep=1.0 林大新 -> 林小新[label="父母關係",weight="0.2"] 張大光 -> 陳小新[label="父母關係",weight="0.2"] 林大新 -> 張大光[label="夫妻關係",weight="0.2"] 張大光 -> 林大新[label="夫妻關係",weight="0.2"] } ``` ---- ## 任務:question: - 找出`Person`和`Movie`節點之間有`DIRECTED`或`ACTED_IN`關係的所有節點 ```sql! MATCH ()-[]->() RETURN ``` ---- ## 任務:100: - 找出`Person`和`Movie`節點之間有`DIRECTED`或`ACTED_IN`關係的所有節點 ```sql! MATCH (p:Person)-[r:DIRECTED|ACTED_IN]->(m:Movie) RETURN p, m ``` --- ## 依距離尋找關係 - 要保人與受益人之間的關係路徑在1到3之間 ```sql! MATCH (p:PolicyHolder)-[*1..3]-(b:Beneficiary) RETURN p, b ``` ```graphviz digraph hierarchy { nodesep=1.0 林大新 -> 林小新[label="父母關係",weight="0.2"] 張大光 -> 陳小新[label="父母關係",weight="0.2"] 林大新 -> 張大光[label="夫妻關係",weight="0.2"] 張大光 -> 林大新[label="夫妻關係",weight="0.2"] } ``` ---- ## 任務:question: - 找出`Person`和`Movie`節點之間關係路徑在1到5之間的所有節點 ```sql! MATCH ()-[]->() RETURN ``` ---- ## 任務:100: - 找出`Person`和`Movie`節點之間關係路徑在1到5之間的所有節點 ```sql! MATCH (p:Person)-[*1..5]->(m:Movie) RETURN p, m ``` ---- ## 內建函數最短距離`shortestPath` - 找出兩個客戶之間的最短關係路徑 ```sql! MATCH p=shortestPath( (a:Customer {id:"A123456789"})-[*]-(b:Customer {id:"B987654321"}) ) RETURN p ``` ```graphviz digraph hierarchy { nodesep=1.0 林小新 -> A123456789[label="父母關係",weight="0.2"] B987654321 -> 林小新[label="夫妻關係",weight="0.2"] 林小新 -> B987654321[label="夫妻關係",weight="0.2"] } ``` ---- ## 任務:question: - 找出`Tom Cruise`和`Meg Ryan`之間的最短關係路徑 ```sql! MATCH p=shortestPath( ()-[*]-() ) RETURN p ``` ---- ## 任務:100: - 找出`Tom Cruise`和`Meg Ryan`之間的最短關係路徑 ```sql! MATCH p=shortestPath( (tom:Person {name:"Tom Cruise"})-[*]-(meg:Person {name:"Meg Ryan"}) ) RETURN p ``` ```graphviz digraph hierarchy { nodesep=1.0 "Tom Cruise" -> "Top Gun"[label="ACTED_IN",weight="0.2"] "Meg Ryan" -> "Top Gun"[label="ACTED_IN",weight="0.2"] } ``` --- ## 依對象與關係尋找所有其他對象 - 找出與要保人`A123456789`共用帳單寄送地址的所有客戶 ```sql! MATCH (p:PolicyHolder {id: "A123456789"})-[:USES_FOR_BILLING]->(:Address)<-[:USES_FOR_BILLING]-(others:Customer) RETURN others.id ``` ```graphviz digraph hierarchy { nodesep=1.0 A123456789 -> 林小新[label="父母關係",weight="0.2"] A123456789 -> 忠孝東路一段一號[label="帳單寄送地",weight="0.2"] 林小新 -> 忠孝東路一段一號 [label="帳單寄送地",weight="0.2"] } ``` ---- ## 任務:question: - 找出與`Tom Cruise`合演過電影(關係類別`ACTED_IN`)的所有演員 ```sql! MATCH ()-[]->()<-[]-() RETURN ``` ---- ## 任務:100: - 找出與`Tom Cruise`合演過電影(關係類別`ACTED_IN`)的所有演員 ```sql! MATCH (:Person {name: "Tom Cruise"})-[:ACTED_IN]->(:Movie)<-[:ACTED_IN]-(coActor:Person) RETURN coActor.name ``` | "coActor.name" | | -------- | | "Jerry O'Connell" | | "Bonnie Hunt" | | "Jay Mohr" | ---- ## SQL vs. Cypher - 找出與`Tom Cruise`合演過電影(關係類別`ACTED_IN`)的所有演員 ```sql! MATCH (:Person {name: "Tom Cruise"})-[:ACTED_IN]->(:Movie)<-[:ACTED_IN]-(coActor:Person) RETURN coActor.name ``` ```sql! WITH tom_movies AS ( SELECT movie.movie_id FROM movie INNER JOIN person_movie ON movie.movie_id = person_movie.movie_id INNER JOIN person ON person_movie.person_id = person.person_id WHERE person.name = "Tom Cruise") SELECT person.name FROM person INNER JOIN person_movie ON tom_movies = person_movie.movie_id INNER JOIN person ON person_movie.person_id = person.person_id INNER JOIN involvement ON person_movie.involve_id = involvement.involve_id WHERE involvement.title = "Actor" ``` --- ## 與SQL類似的語法 - `AS` {name}: 變數別名 - `ORDER BY` {attribute}: 根據屬性排序 - `LIMIT` {number}: 限制返回結果數 ---- ## 排序 - 尋找名字(`givenName`)為`小新`的所有顧客節點(`Customer`)，根據姓(`familyName`)排序，取前三個 ```sql! MATCH (c:Customer) WHERE c.givenName = '小新' RETURN c.familyName AS family_name, c.givenName AS given_name ORDER BY family_name LIMIT 3 ``` | family_name | given_name | | -------- | -------- | | 林 | 小新 | | 陳 | 小新 | | 張 | 小新 | ---- ## 任務:question: - 找出標籤為`Movie`的節點並且根據發行年份(屬性名稱`released`)排序，同時列出電影名稱與發行年份，取前三個結果 ```sql! MATCH () RETURN ORDER BY LIMIT ``` ---- ## 任務:100: - 找出標籤為`Movie`的節點並且根據發行年份(屬性名稱`released`)排序，同時列出電影名稱與發行年份，取前三個結果 ```sql! MATCH (m:Movie) RETURN m.title AS title, m.released AS year ORDER BY year LIMIT 3 ``` ```txt= title year "One Flew Over the Cuckoo's Nest" 1975 "Top Gun" 1986 "Stand By Me" 1986 ``` --- ## 寫入DB - 小量資料: 手動寫入 - 中量資料: 使用[Neo4j Data Importer](http://data-importer.graphapp.io/)匯入CSV - 大量資料: - 使用Cypher匯入CSV - 使用[Neo4j ETL Tool](https://neo4j.com/labs/etl-tool/) ---- ## 手動寫入 - 創建兩個標籤為`Person`的節點 ```sql= MERGE (h:Person {name: "Howard"}) MERGE (a:Person {name: "Alice"}) ``` - 使用迴圈 ```sql= WITH ['Howard', 'Alice'] AS names FOREACH ( value IN names | MERGE (:Person {name: value}) ) ---- - 創建關係`FOLLOWS` ```sql= MATCH (h:Person {name: "Howard"}) MATCH (a:Person {name: "Alice"}) MERGE (h)-[r:FOLLOWS]->(a) ``` :no_entry_sign: Neo4j[官方建議](https://neo4j.com/developer/guide-import-csv/)不要同時創建節點與關係 :+1:最佳做法為**創建節點之後再連結節點之間的關係** ---- ## 匯入CSV - 範例: [Northwind資料集](https://relational.fit.cvut.cz/dataset/Northwind) - CSV下載點`https://data.neo4j.com/northwind` - `/products.csv` - `/categories.csv` - `/suppliers.csv` - `/orders.csv` - `/customers.csv` - `/order-details.csv` ---- - 使用[Neo4j Data Importer](http://data-importer.graphapp.io/)匯入  ---- - 使用Cypher匯入 ```sql :play northwind graph ``` - 將下載的CSV檔放到專案路徑 - :exclamation: 將`https://`改成`file:///` ---- ## 結語 - [金融防詐識別](https://github.com/neo4j-contrib/gists/blob/master/other/BankFraudDetection.adoc)應用案例 ---- ## 官方推薦書籍  [下載](https://graphdatabases.com/) ---- ## 官方推薦書籍  [下載](https://neo4j.com/book-learning-neo4j/)