GITBOOK-35: No subject

Author: Filip

docs/api/rest-api-reference/README.md

@@ -1,2 +1,3 @@
-# API Reference
+# REST API Reference
 
+This is the reference for the REST API part of Robopipe API. To try out particular endpoints, simply navigate to the appropriate endpoint and click on the "**Test it**" button. All endpoints have included the request and response body schema, along with the restrictions for particular attributes. When running the API, you can also open your locally running docs on [https://robopipe-1.local/docs](https://robopipe-1.local/docs) (Swagger UI). The OpenAPI specification can be downloaded from your server (controller) at [https://robopipe-1.local/openapi.json](https://robopipe-1.local/openapi.json) or below.

docs/api/rest-api-reference/streams.md

@@ -1,6 +1,14 @@
 # Streams
 
-Each camera contains a number of sensors of different types. Our API currently supports manipulation of color and monochromatic image sensors. Each sensor produces image data in various formats and is configurable independently of other sensor on the same device. Sensors are addressed by their socket name which is in the format [_CAM\_(A-H)_](#user-content-fn-1)[^1]_._
+Each camera contains a number of sensors of different types. Each sensor (or combination of multiple sensors) provides an image **stream** which can be consumed directly as video stream, or fed as input into a neural network. Our API exposes operations through which these **streams** can be consumed or manipulated. Each stream is addresses either by its sensor name  - CAM\_(A-H)[^1] or by its combined name - DEPTH\_(A-H)\_(A-H)[^2]_._
+
+## Depth API
+
+Some camera models containing multiple image sensors, support a feature called _Stereo Depth_. This feature combines data from two image sensors at a time, to calculate the depth of the image in front of it. The result is a single image stream. The image will consist of depth data calculated by the camera, lighter areas will contain objects that are closer to the camera while darker areas will contain objects further away. This stream is configurable the same way as a single sensor.
+
+{% hint style="info" %}
+Depth stream and the respective sensors it uses cannot be both active at the same time. E.g. if stream DEPTH\_B\_C is active, then streams CAM\_B and CAM\_C will be inactive and vice versa.
+{% endhint %}
 
 ## API Reference
 
@@ -40,4 +48,6 @@ Each camera contains a number of sensors of different types. Our API currently s
 
     available sensor, with names CAM\_A,
 
-    CAM\_B, CAM\_C, etc.
+    CAM\_B, CAM\_C, ... depending on you camera model.
+
+[^2]: The first and second capital letters A-H in the stream name correspond to the left and right image sensor respectively.

docs/api/websocket-api-reference/README.md

@@ -1,2 +1,3 @@
 # WebSocket API Reference
 
+Along with the REST API, Robopipe API also exposes some WebSocket endpoints for cosuming live data. Particularly video streams and neural network detections. You can find the specifications at their respective pages below.

docs/api/websocket-api-reference/nn-inference.md

@@ -2,7 +2,7 @@
 
 ## Running inference
 
-The neural network starts at the point it is uploaded to the camera. The output from the model can be read by subscribing to a websocket endpoint `ws://host:port/cameras/{mxid}/sensors/{sensor_name}/nn`. The outputted data will be a json in the format:
+The neural network starts at the point it is uploaded to the camera. The output from the model can be read by subscribing to a websocket endpoint `ws://host:port/cameras/{mxid}/streams/{sensor_name}/nn`. The outputted data will be a json in the format:
 
 ```json
 {
@@ -11,3 +11,43 @@ The neural network starts at the point it is uploaded to the camera. The output
 ```
 
 where detections will contain whetever you is your model's output.
+
+## Output format
+
+The data format received from the websocket endpoint depends on the type of neural network you have running on the camera.
+
+### Generic neural network
+
+In case you used a generic neural network, the output data will be a one dimensional array containing raw data from the last layer of the neural network.
+
+```json
+{
+    "detections": [
+        0.3,
+        0.02,
+        0.05,
+        ...
+    ]
+}
+```
+
+### YOLO & MobilenetSSD
+
+In case you have chosen either a YOLO or MobilenetSSD model, the output format will be an array containing detection objects. Each detection object contains:
+
+* label - number indicating the label it detected
+* confidence - float in the range \[0, 1] indicating the confidence level of the detection
+* coords - four element tuple, containing the coordinates of the detection in the image \[x\_min, y\_min, x\_max, y\_max]
+
+```json
+{
+    "detections": [
+        {"label": 0, "confidence": 0.765, "coords": [0.5, 0.1, 0.6, 0.23]},
+        ...
+    ]
+}
+```
+
+#### Depth
+
+In case a depth stream is active and running at the same time a neural network is deployed, each detection object will also contain information regarding its depth position.

docs/api/websocket-api-reference/stream.md

@@ -6,7 +6,7 @@ Each sensor provides its own video stream. If there are no subscribers, no data
 
 ### Receiving segments
 
-The data obtained via websockets is MP4 fragments encoded using H.264 codec. To start receiving data, simply open a websocket at `ws://host:port/camera/MXID/sensors/SENSOR_NAME/video`.
+The data obtained via websockets is MP4 fragments encoded using H.264 codec. To start receiving data, simply open a websocket at `ws://host:port/camera/MXID/streams/SENSOR_NAME/video`.
 
 ### Playing the video