Queries API

Queries API allows you to retrieve SQL queries executed on your cluster.

List of Queries

Retrieve queries that happen today

Retrieves queries that happened today.

CODE

GET /api/monitor/get/queries

Retrieve all queries since

Retrieve all queries since a specified unix timestamp. This is particularly useful to get an incremental listing, that displays modifications you made since your last listing. Each listing will give you a JSON response with a last value:

CODE

GET /api/monitor/get/lastqueries/:time

You can use this value for future queries.

Retrieve all queries at the specified date

Retrieve all queries at the specified date. Parameter date is in yyyy-mm-dd format.

CODE

GET /api/monitor/get/queries/:date

Retrieve all queries executed on the specified period

Retrieve all queries executed on a specific period. Parameters start and end are UNIX timestamp.

CODE

GET /api/monitor/get/lastqueries/:start/:end

JSON Format

All queries have the same JSON structure for the above-mentioned endpoints.

Example

JS

{
    "runnings": [],
    "last": 1574764869963,
    "reqs": [
        {
            "sql": "SELECT ( CAST( ( CASE WHEN 2 = 0 THEN NULL ELSE `demo_nyc_cab`.`trip_distance` / 2 END ) AS BIGINT ) * 2 ) AS `calculation_4389883802952822784`, AVG(`demo_nyc_cab`.`tip_amt`) AS `avg_tip_amt_ok`, YEAR(`demo_nyc_cab`.`trip_pickup_datetime`) AS `yr_trip_pickup_datetime_ok` FROM etalon.nyc_yellow_full `demo_nyc_cab` LEFT JOIN etalon.nyc_cab_dim_dest p ON start_lat = p.dest_latitude GROUP BY ( CAST( ( CASE WHEN 2 = 0 THEN NULL ELSE `demo_nyc_cab`.`trip_distance` / 2 END ) AS BIGINT ) * 2 ), YEAR(`demo_nyc_cab`.`trip_pickup_datetime`)",
            "operation": "SELECT",
            "submit": 1,
            "finished": 1,
            "perf": 1607,
            "date": 1574764806362,
            "error": "",
            "warning": "",
            "id": -1,
            "killable": true,
            "killed": false,
            "load": false,
            "nb": 26,
            "user": "hadoop",
            "ip": "127.0.0.1",
            "run": false,
            "cached": false,
            "delegated": false,
            "elapsed": 1607,
            "pending_start": -1,
            "pending_end": -1,
            "request_evictions": 0,
            "query_evictions": 0,
            "max_worker_memory": 4259,
            "request_master_memory": 5611,
            "read_lines_from_buckets": 3180983,
            "disk_usage": 0,
			"pending": false,
 	        "pending_duration": 0,
			"has_spill": false,
			"freezing": false,
        	"freeze_duration": 4456,
			"bigindex": false,
            "indexInformations": [
                {
                    "tableName": "sansprejoin1",
                    "computedJoins": 0,
                    "cols": [
                        {
                            "name": "trip_distance",
                            "hit": 2
                        },
                        {
                            "name": "end_lat_round3",
                            "hit": 0
                        }
                    ]
                }
            ]
        }
    ]
}

Description

Name	Type	Description
sql	String	SQL query executed.
operation	String	Type of SQL query.
submit	Integer	Number of tasks submitted to execute the query.
finished	Integer	Number of tasks finished.
date	Timestamp	Execution date of the query in a Unix timestamp.
error	String	Error message if the query fails.
killable	Boolean	True if the query is killable.
killed	Boolean	True if the query has been killed.
user	String	User who runs the query.
ip	String	IP Address of the user executing the query.
run	Boolean	True if the query is running.
cached	Boolean	True if the query comes from the cache.
delegated	Boolean	True if the query has been delegated to an external system.
elapsed	Integer	Execution time of the query, in ms.
max_worker_memory	Integer	Amount of memory used on maximum to compute the query before aggregation, on bytes.
request_master_memory	Integer	Amount of memory used by the master to aggregate result, on bytes.
read_lines_from_buckets	Integer	Number of lines read from bucket for the query.
disk_usage	Integer	Amount of disk space used if the query has spilled on disk.
indexInformations.tableName	String	Name of the index used to execute this query.
indexInformations.computedJoins	Integer	Number of computed joins.
indexInformations.cols.name	String	Name of the column in the index used.
indexInformations.cols.hit	Integer	Each time this column has been referenced in the query.
pending	Boolean	True is the query is in pending state (waiting for an execution slot), false otherwise.
pending_duration	Integer	Total time (ms) while the query was in a pending state.
has_spill	Boolean	True is the query used the spill-to-disk feature, false otherwise.
freezing	Boolean	True is the query is in freezing state (the query is started and flagged as big-request and waiting for a big-request slot) false otherwise.
freeze_duration	Integer	Total time (ms) while the query was in a freezing state.
bigindex	Boolean	True is the query used the bigindex feature, false otherwise.

SQL Operations

The following list describes the SQL operation available :

SELECT
LOAD
CREATE
INSERT
DELETE
DROP
TRUNCATE
SHOW_MEMORY
SHOW_TABLECOUNT
SHOW_DICTIONARIES
COMMIT
ROLLBACK
PAUSE
DESCRIBE
ALTER_DDL
ADD_INDEX
DROP_INDEX
SYNCHRONIZE
SET
CREATE_SCHEMA
DROP_SCHEMA
USE
EXPORT
IMPORT
CREATE_VIEW
DROP_VIEW