Module: queue_worker

The queue worker is the module responsible to retrieve data from the S3 bucket based on the parameters provided. It fetches a new request from the request queue interface (Rabbitmq) and does the processing. As soon data is downloaded from S3 it passes the data to the Data Extractor module which is the submodule of queue worker. Data Extractor converts data in the desired format required for the system to work and then passes back the formatted data to the queue worker. The queue worker on retrieval of the desired format from the submodule makes an API call to the DB Middleware module to store it in the database. The queue worker uses an endpoint exposed by the Db MiddleWare. The details of the endpoint are as follows

POST /data_writer

Request:

{
    "requestID": "KAKQ_02012022_003000_005959_Reflectivity",
    "start_time": "body",
    "end_time": "end_time",
    "data": {
        "stationName": "stationName",
        "stationLatitute": "stationLatitute",
        "stationLongitude": "stationLongitude",
        "stationID": "stationID",
        "property": "Reflectivity",
        "latitude": [
            ["d"],
            ["e"],
            ["f"]
        ],
        "longitude": [
            ["a"],
            ["b"],
            ["c"]
        ],
        "Reflectivity": [
            ["x"],
            ["y"],
            ["x"]
        ]
    }
}

Response:

{ 
status: "success/error", 
message: "Insertion successful/Insertion failed" 
}