The titan Repository Service
The titan Repository Service is in charge of communicating with a database. It can be reached via a REST API that serves under the port number HOST:8085.
Building/ running
To run the service simply use go run ./cmd/repository
or to create an exe go build ./cmd/repository
It features the following command line arguments:
| Name | Options (default) | Purpose |
|---|---|---|
| port | (8085) | Port on which the repository serves |
| database | (127.0.0.1:27017) | Host:port of the database server |
| log-level | info/debug/warning/error | Log level |
| (info) | ||
| log-to-kafka | (false) | Flag: use kafka hook for logging |
| kafka | (10.14.0.23:9092) | host:port of kafka broker |
| service-topic | (titan.servicelogs) | kafka topic for logs |
Generally, all settings listed above can be done also via environment variables, whereby the variables are composed of a prefix (TITAN_RS) and the argument using snake case:
TITAN_RS_ARGUMENT, e.g. TITAN_RS_LOG_LEVEL
Currently, a MongoDB client is implemented, which expects a Mongo DB server at database.
For testing e.g. set up a test MongoDB server using:
docker pull mongo
docker run -d -p 27017:27017 --name mongodb mongo
Usage
Generally, all queries to the repository service are formulated by sending a json object with the fields:
| Name | Type | Doc | Constraints |
|---|---|---|---|
| database | string | optional database name | |
| collection | string | optional collection name | |
| find | object | optional find query for a selection | |
| document | string | optional document ID | |
| value | object | json parsable object that is to be stored |
The operations are differentiated by which fields you are filling with values.
document: "DocumentName",
database: "DatabaseName",
collection: "CollectionName"
find: FindQueryObject
value: Object
The repository is structured in named Databases that contain Collections, which hold Values in Documents. In the response, you either get only a status code or a json object containing the value you queried.
Inserting values to a database
Currently, only inserting individual documents is supported.
Inserting an object to the DB can be achieved by a PUT request to:
endpoint: /object/ Type: PUT
The request has to be a json object following the structure described in RequestScheme.md, but you can omit the entry "find".
In the database, this will create a new document with the ID given in the "document" entry, in the collection given in "collection" which is part of the database "database". The object to be inserted is given in the "value" field. In case of a successful insertation you will the http Status Ok in the response, otherwise you will get an status internal server error.
Example
To save the configuration of a flow(see Resources.md) in the storage service in a database called configuration, in the collection flows, send a flow object with the name "Demo" (see Resources.md) in a PUT request with a json file that contains:
document: "UUID",
database: "configuration",
collection: "flows"
value: Flow
which corresponds to the MongoDB query:
db.configuration.insertOne({
_id: "UUID",
value: Flow,
});
Updating values in a database
Currently, only updating complete documents is supported.
Updating an object to the DB can be achieved by a PATCH request to:
endpoint: /object/ Type: PATCH
The request has to be a json object following the structure described in RequestScheme.md, but you can omit the entry "find".
In the database, this will the document with the ID given in the "document" entry, in the collection given in "collection" which is part of the database "database". It will replace the document by the data given in "value". In case of a successful update you will the http Status Ok in the response, otherwise you will get an status internal server error.
Example
To update the configuration of a flow(see Resources.md) in the storage service in a database called configuration, in the collection flows, send an updated flow object with the name "Demo" :
document: "UUID",
database: "configuration",
collection: "flows"
value: Flow
which corresponds to the MongoDB query:
db.configuration.insertOne({
_id: "UUID",
value: Flow,
});
Selecting values from a database
Currently, selecting complete documents and selecting using a query string is supported.
Selecting an object to the DB can be achieved by a GET request to:
endpoint: /object/ Type: GET
The request has to be a json object following the structure described in RequestScheme.md, but you can omit the entry "value!
To select whole documents you have to specify the document to be selected and omit the find element, whereas if you want to subsample from the document you have to omit the document element, but have to give the find element (see example below).
I.e., selecting a whole document corresponding to an SQL query:
SELECT * from collectionName
or the Mongo DB query:
db.getCollection(collectionName).find({});
has to be expressed as :
database: "databaseName",
collection: "collectionName"
The find element consists of a json parsable object, i.e. uses keys and values. If the value is not a single element but a list, the find query will return the document that contains an element matching one of the given list elements. If you do not provide a "document" element and also leave the "find" element empty all documents from the collection are returned. I.e. using a FindQueryObjects results in queries of the form:
SELECT * from collectionName WHERE FindQueryObjectKey = FindQueryObjectValue
In the database, the document with the ID given in the "document" entry, in the collection given in "collection" which is part of the database "database" will be selected. In case of a successful update you will the http Status Ok in the response, otherwise you will get an status internal server error. Furthermore, in the body of the response you will get the "value", i.e. the object you requested.
Example
To retrieve the configuration of a flow called Demo from the storage service from a database called configuration, of the collection flows, send a GET request with the json object:
document: "UUID",
database: "configuration",
collection: "flows"
which would correspond to the MongoDB query in the database 'configuration':
db.getCollection("flows").findOne({ id: "UUID" });
In the response you will get the flow object that was previously stored there as a json object.
To get all documents from the database configuration of the collection flows you send a GET request with the following content:
database: "configuration",
collection: "flows"
which would correspond to the MongoDB query in the database 'configuration':
db.getCollection("flows").find({});
To get all documents that contain running flows from the database state:
database: "state",
collection: "flows"
find: "Running" : false
which would correspond to the MongoDB query in the database 'state':
db.getCollection("flows").find({ "value.Running": false });
Deleting values from a database
Currently, only deleting complete documents is supported.
Inserting an object to the DB can be achieved by a DELETE request to:
endpoint: /object/ Type: DELETE
The request has to be a json object following the structure described in RequestScheme.md, but you can omit the entries "value" and "find"
In the database, the document with the ID given in the "document" entry, in the collection given in "collection" which is part of the database "database" will be deleted. In case of a successful update you will the http Status Ok in the response, otherwise you will get an status internal server error.
Example
To delete the configuration of a flow called Demo from the storage service from a database called configuration, of the collection flows, send a DELETE with the object:
document: "UUID",
database: "configuration",
collection: "flows"