Dima/orion_greptime_proto
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: promql service (#26)

Browse Source 
* feat: promql service

* feat: promql service

* feat: prometheus service, and common.proto, prom.proto

* feat: add header in prom request/response

* docs: include new proto files in readme
This commit is contained in:
yuanbohan
2023-04-21 14:29:48 +08:00
committed by GitHub
parent 0bebe5f69c
commit fd18fe5ca3
9 changed files with 1314 additions and 758 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+47 -16
View File
@@ -6,7 +6,7 @@ GreptimeDB protobuf files.
### Requirement
- [google/protobuf](https://github.com/protocolbuffers/protobuf) v3
- [google/protobuf][protobuf] v3
### Command
@@ -15,13 +15,13 @@ GreptimeDB protobuf files.
```console
make rust
```
- **Compile for Go**
```console
make go
```
The compilation will use builder container `namely/protoc-all`.
## Usage
@@ -63,33 +63,64 @@ import (
## For SDK developers
GreptimeDB's gRPC service is built on top of [Arrow Flight RPC](https://arrow.apache.org/docs/format/Flight.html). You can find the Arrow's official implementation status of each programming language [here](https://arrow.apache.org/docs/status.html#flight-rpc).
GreptimeDB's gRPC service is built on top of [Arrow Flight RPC][flight]. You can find the Arrow's
official implementation status of each programming language [flight rpc][flight-rpc].
> If you can't find the language you are using, you can always generate the Arrow Flight gRPC service from the raw protobuf definition [here](https://arrow.apache.org/docs/format/Flight.html#protocol-buffer-definitions). Or call into other language binding like C++.
> If you can't find the language you are using, you can always generate the Arrow Flight gRPC
> service from the raw protobuf definition [flight protobuf definitions][flight-protobuf]. Or call
> into other language binding like C++.
Once the Arrow Flight client is ready, you only need to care about the following 3 protobuf files to accomplish our SDK writing:
Once the Arrow Flight client is ready, you only need to care about the following protobuf files to
accomplish our SDK writing:
```console
.
├── greptime
│   └── v1
│   ├── column.proto
│   ├── database.proto
│   ├── ddl.proto
│   ├── column.proto
│   ├── common.proto
│   ├── database.proto
│   ├── ddl.proto
│   ├── health.proto
│   └── prom.proto
```
> You can find all protobuf files in the directory "proto" under the project's root.
Right now the GreptimeDB only responds to Arrow Flight's `do_get` interface. All the reads and writes (that are from clients) are handled there. `do_get` needs a "ticket" as the input request, you need to serialize the `GreptimeRequest` message defined in "database.proto" for it.
Right now the GreptimeDB only responds to Arrow Flight's `do_get` interface. All the reads and
writes (that are from clients) are handled there. `do_get` needs a "ticket" as the input request,
you need to serialize the `GreptimeRequest` message defined in "database.proto" for it.
There are 3 kinds of `GreptimeRequest`, which are:
- `InsertRequest`, carries the data to be ingested. It's a little verbose to assemble, especially the "column"s that define the schema and value of the input data. You can find the definition of "column" in "column.proto".
- `QueryRequest` has the SQL in it. Note that you can `INSERT INTO` GreptimeDB as well as `SELECT` it.
- `DdlRequest` defines the "Data Definition Language" request, like table creation or deletion. It's sometimes more representative than the normal SQL. `DdlRequest`s are defined in "ddl.proto".
- `InsertRequest`, carries the data to be ingested. It's a little verbose to assemble, especially
the "column"s that define the schema and value of the input data. You can find the definition of
"column" in "column.proto".
- `QueryRequest` has the SQL in it. Note that you can `INSERT INTO` GreptimeDB as well as `SELECT`
it.
- `DdlRequest` defines the "Data Definition Language" request, like table creation or deletion. It's
sometimes more representative than the normal SQL. `DdlRequest`s are defined in "ddl.proto".
There's also `RequestHeader` in the `GreptimeRequest` to specify the "catalog" and "schema" to be used in this request. If either one is not set (or left empty), GreptimeDB will use the default catalog "greptime" and default schema "public".
There's also `RequestHeader` in the `GreptimeRequest` to specify the "catalog" and "schema" to be
used in this request. If either one is not set (or left empty), GreptimeDB will use the default
catalog "greptime" and default schema "public".
The response of `do_get` is handled in Arrow Flight's client. It's a stream of `FlightData`. You can find its definition in Arrow Flight's protobuf file. Special care must be taken when dealing with insertion, that GreptimeDB puts insertion result in `FlightData`'s metadata. Insertion results, either from `InsertRequest` or `INSERT INTO` SQL, are both have the same format, that "Affected Rows: x". "x" represents the rows that are successfully inserted. When dealing with this special `FlightData`, please ignore its `data_body` field, but directly strip the `app_metadata` field from it, and deserialize the bytes as `FlightMetadata` message. You will find the "affected rows" result in it.
The response of `do_get` is handled in Arrow Flight's client. It's a stream of `FlightData`. You can
find its definition in Arrow Flight's protobuf file. Special care must be taken when dealing with
insertion, that GreptimeDB puts insertion result in `FlightData`'s metadata. Insertion results,
either from `InsertRequest` or `INSERT INTO` SQL, are both have the same format, that "Affected
Rows: x". "x" represents the rows that are successfully inserted. When dealing with this special
`FlightData`, please ignore its `data_body` field, but directly strip the `app_metadata` field from
it, and deserialize the bytes as `FlightMetadata` message. You will find the "affected rows" result
in it.
We already have our SDK written in [Java](https://github.com/GreptimeTeam/greptimedb-client-java), [Rust]() and [Go](), feel free to take any of them as an example.
We already have our SDK written in [Java][java-sdk], [Rust]() and [Go][go-sdk], feel free to take
any of them as an example.
<!-- links -->
[protobuf]: https://github.com/protocolbuffers/protobuf
[flight]: https://arrow.apache.org/docs/format/Flight.html
[flight-rpc]: https://arrow.apache.org/docs/status.html#flight-rpc
[flight-protobuf]: https://arrow.apache.org/docs/format/Flight.html#protocol-buffer-definitions
[java-sdk]: https://github.com/GreptimeTeam/greptimedb-client-java
[go-sdk]: https://github.com/GreptimeTeam/greptimedb-client-go