or
or
By clicking below, you agree to our terms of service.
New to HackMD? Sign up
Syntax | Example | Reference | |
---|---|---|---|
# Header | Header | 基本排版 | |
- Unordered List |
|
||
1. Ordered List |
|
||
- [ ] Todo List |
|
||
> Blockquote | Blockquote |
||
**Bold font** | Bold font | ||
*Italics font* | Italics font | ||
~~Strikethrough~~ | |||
19^th^ | 19th | ||
H~2~O | H2O | ||
++Inserted text++ | Inserted text | ||
==Marked text== | Marked text | ||
[link text](https:// "title") | Link | ||
 | Image | ||
`Code` | Code |
在筆記中貼入程式碼 | |
```javascript var i = 0; ``` |
|
||
:smile: | ![]() |
Emoji list | |
{%youtube youtube_id %} | Externals | ||
$L^aT_eX$ | LaTeX | ||
:::info This is a alert area. ::: |
This is a alert area. |
On a scale of 0-10, how likely is it that you would recommend HackMD to your friends, family or business associates?
Please give us some advice and help us improve HackMD.
Do you want to remove this version name and description?
Syncing
xxxxxxxxxx
Everything you need to know about Nebula Graph Index
The term Index in Nebula Graph is quite similar to the same term in relational databases, but they are not exactly the same. I noticed that some Nebula Graph users are often confused when getting started with Nebula Graph. Typically, people want to know what exactly Nebula Graph Index is, when should they use it, and how it impacts the performance of Nebula Graph.
Today I'm going to walk you through the Index concept in Nebula Graph and hopefully, this article will answer these questions.
Let's get started!
What exactly Nebula Graph Index is
To put it short, Nebula Graph Index is only used when the vertextID is not specified, or only when properties of vertices or edges are defined in the query conditions.
Index is only used in a starting entry of a graph query. If a query is in the pattern: (a->b->c, where c is with the condition "foobar"), since the only filter
condition-foobar
is onc
, this query under the hood will start to look forc
, and then it walks reversely through the->
tob
, and finally toa
. Thus, the Nebula Graph Index will be used and only be possibly used when locating c.Index is used only to seek starting points
We know that in RDBMS, indexing is to create a duplicate of sorted data to enable QUERY with conditional filtering on the sorted data, in order to accelerate the query in reads and it also brings additional data writes.
In Nebula Graph, the index is to create a duplicate of sorted Vertex/Edge PROP DATA to locate the starting point of a QUERY.
Not all of queries relied on the index, here are some example queries, where the starting points are only defined using conditions, rather than VertextIDs. Let's call them
pure property condition starting queries
:In both
query 0
andquery 1
, the pattern is to "Find VID/EDGE only based on given property conditions".On the contrary, the starting point is VertexID based instead in
query 2
andquery 3
:If we look into
query 1
andquery 3
, which shared the same condition on vertices ontag:player
, which are both{ name: 'Tim Duncan' }
, they are differentiated in starting points:For
query 3
, the index is not required as the query will start from the known vertex ID in["player101", "player102"]
and thus:v2
's vertex IDsv2
, GetVertices() for the next hop:v
and filter based on the property:name
For
query 1
, the query has to start fromv
due to no known vertex IDs were provided:{ name: 'Tim Duncan' }
v
, GetVertices() for the next hop:v2
Now, we know the whole point that matters here is on whether we know the vertexID. And the above differences could be shown in their execution plans with PROFILE or EXPLAIN like the following:
query 1
, requires index(on tag: player), pure property condition query as starting pointquery 3
, no index required, query starting from known vertex IDsWhy Nebula Graph index is an enabler rather than an accelerator
Can't those queries be done without indexes?
It's possible in theory with a full scan, but disabled without an index.
The reason is that Nebula Graph stores data in a distributed and graph-oriented way, the full scan of data was considered too expensive to be allowed.
Why starting point only
Index data is not used in the traversal. It could confuse us to think of index as sorting data based on properties, does it accelerate the traversal with property condition filtering? The answer is no.
In Nebula Graph, the data is structured in a way to enable fast graph-traversal, which is already indexed/sorted on vertex ID(for both vertex and edge) in the raw data, where traversal(underlying in storage, it's calling GetNeighbors interface) of the given vertex is cheap and fast due to the persistent storage.
So in summary:
Facts on Nebula Graph Index
To understand more details/limitations/cost of Nebula, let's reveal more about its design. Here are some facts in short:
Index Data is stored and shared together with Vertex Data
It's Left Match based only: It's RocksDB Prefix Scan under the hood
Effect on write and read path(to see its cost):
Data Full Scan LIMIT Sample(not full scan) is supported without index
LOOKUP ON t YIELD t.name | LIMIT 1
The key info can be seen in one of my sketch notes:
Within this sketch note, more highlights are:
It's a Local Index Design
There are costs in the writing path
For the reading path:
Takeaways:
How to use the index
We should always refer to the documentation, and I just put some highlights on this here:
To create an index on a tag or edge type to specify a list of props in the order that we need.
CREATE INDEX
If an index was created after existing data was inserted, we need to trigger an index asynchronously to rebuild the job, as the index data will be written in a synchronous way only when the index is created.
REBUILD INDEX
We can see the index status after
REBUILD INDEX
is issued.SHOW INDEX STATUS
Queries levering index could be LOOKUP, and with the pipeline, in most cases, we will do follow-up graph-walk queries like:
Or in MATCH query like this, under the hood, v will be searched on index and v2 will be walked by default graph data structure without involving index.
Recap
Finally, Let's Recap
CREATE INDEX
on existing dataHappy Graphing!
Feture image credit to Alina