Storm UI REST API
Storm UI 守护程序提供了一个 REST API, 允许您与 Storm 集群进行交互, 其中包括检索 metrics（度量）数据和配置信息, 启动或停止 topologies（拓扑）的管理操作。
数据格式
该 REST API 返回 JSON 响应并支持 JSONP. 客户端可以传回一个回调查询参数, 以在回调函数中包装 JSON。 REST API allows CORS by default.
使用 UI REST API
Note（注意）: 建议忽略 JSON 响应中的 undocumented elements（未记录元素），因为未来版本的 Storm 可能不再 支持这些元素.
REST API Base URL
REST API 是 Storm 的 UI 守护进程（由 "storm ui" 启动）的一部分，因此与 Storm UI 在同一主机和端口上运行（UI 守护程序通常与 Nimbus 守护程序在同一主机上运行）. 通过 ui.port 来配置 port（端口）, 它默认设置为 8080（请参阅 defaults.yaml）.
该 API 的 base URL 将是:
http://<ui-host>:<ui-port>/api/v1/...
您可以使用 curl 这样的工具来操作 REST API:
# 请求集群配置
# Note（注意）: 我们假设 ui.port 配置的默认值是 8080.
$ curl http://<ui-host>:8080/api/v1/cluster/configuration
在安全的环境中模仿用户
在安全环境中, 经过身份验证的用户可以模拟另一个用户. 为了模拟用户, 调用者必须通过 doAsUser 参数或 header, 其值设置为用户需要执行该 request 请求. 请看 SECURITY.MD 以了解有关如何设置模拟 ACL 和授权的更多信息. 其余的 API 使用与 nimbus 使用的相同的配置和 acls.
示例:
1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1425844354\?doAsUser=testUSer1
2\. curl 'http://localhost:8080/api/v1/topology/wordcount-1-1425844354/activate' -X POST -H 'doAsUser:testUSer1'
GET Operations
/api/v1/cluster/configuration (GET)
Returns the cluster configuration.
Sample response (does not include all the data fields):
{ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs": null, "topology.builtin.metrics.bucket.size.secs": 60, "topology.fall.back.on.java.serialization": true, "topology.max.error.report.per.interval": 5, "zmq.linger.millis": 5000, "topology.skip.missing.kryo.registrations": false, "storm.messaging.netty.client_worker_threads": 1, "ui.childopts": "-Xmx768m", "storm.zookeeper.session.timeout": 20000, "nimbus.reassign": true, "topology.trident.batch.emit.interval.millis": 500, "storm.messaging.netty.flush.check.interval.ms": 10, "nimbus.monitor.freq.secs": 10, "logviewer.childopts": "-Xmx128m", "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size": 1024, }
/api/v1/cluster/summary (GET)
Returns cluster summary information such as nimbus uptime or number of supervisors.
Response fields:
| Field | Value | Description |
|---|---|---|
| stormVersion | String | Storm version |
| supervisors | Integer | Number of supervisors running |
| topologies | Integer | Number of topologies running |
| slotsTotal | Integer | Total number of available worker slots |
| slotsUsed | Integer | Number of worker slots used |
| slotsFree | Integer | Number of worker slots available |
| executorsTotal | Integer | Total number of executors |
| tasksTotal | Integer | Total tasks |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
| totalMem | Double | The total amount of memory in the cluster in MB |
| totalCpu | Double | The total amount of CPU in the cluster |
| availMem | Double | The amount of available memory in the cluster in MB |
| availCpu | Double | The amount of available cpu in the cluster |
| memAssignedPercentUtil | Double | The percent utilization of assigned memory resources in cluster |
| cpuAssignedPercentUtil | Double | The percent utilization of assigned CPU resources in cluster |
Sample response:
{ "stormVersion": "0.9.2-incubating-SNAPSHOT", "supervisors": 1, "slotsTotal": 4, "slotsUsed": 3, "slotsFree": 1, "executorsTotal": 28, "tasksTotal": 28, "schedulerDisplayResource": true, "totalMem": 4096.0, "totalCpu": 400.0, "availMem": 1024.0, "availCPU": 250.0, "memAssignedPercentUtil": 75.0, "cpuAssignedPercentUtil": 37.5 }
/api/v1/supervisor/summary (GET)
Returns summary information for all supervisors.
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Supervisor's id |
| host | String | Supervisor's host name |
| uptime | String | Shows how long the supervisor is running |
| uptimeSeconds | Integer | Shows how long the supervisor is running in seconds |
| slotsTotal | Integer | Total number of available worker slots for this supervisor |
| slotsUsed | Integer | Number of worker slots used on this supervisor |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
| totalMem | Double | Total memory capacity on this supervisor |
| totalCpu | Double | Total CPU capacity on this supervisor |
| usedMem | Double | Used memory capacity on this supervisor |
| usedCpu | Double | Used CPU capacity on this supervisor |
Sample response:
{ "supervisors": [ { "id": "0b879808-2a26-442b-8f7d-23101e0c3696", "host": "10.11.1.7", "uptime": "5m 58s", "uptimeSeconds": 358, "slotsTotal": 4, "slotsUsed": 3, "totalMem": 3000, "totalCpu": 400, "usedMem": 1280, "usedCPU": 160 } ], "schedulerDisplayResource": true }
/api/v1/nimbus/summary (GET)
Returns summary information for all nimbus hosts.
Response fields:
| Field | Value | Description |
|---|---|---|
| host | String | Nimbus' host name |
| port | int | Nimbus' port number |
| status | String | Possible values are Leader, Not a Leader, Dead |
| nimbusUpTime | String | Shows since how long the nimbus has been running |
| nimbusUpTimeSeconds | String | Shows since how long the nimbus has been running in seconds |
| nimbusLogLink | String | Logviewer url to view the nimbus.log |
| version | String | Version of storm this nimbus host is running |
Sample response:
{ "nimbuses":[ { "host":"192.168.202.1", "port":6627, "nimbusLogLink":"http:\/\/192.168.202.1:8000\/log?file=nimbus.log", "status":Leader, "version":"0.10.0-SNAPSHOT", "nimbusUpTime":"3m 33s", "nimbusUpTimeSeconds":"213" } ] }
/api/v1/history/summary (GET)
Returns a list of all running topologies' IDs submitted by the current user.
Response fields:
| Field | Value | Description |
|---|---|---|
| topo-history | List | List of Topologies' IDs |
Sample response:
{ "topo-history":[ "wc6-1-1446571009", "wc8-2-1446587178" ] }
/api/v1/supervisor (GET)
Returns summary for a supervisor by id, or all supervisors running on a host.
Examples:
1\. By host: http://ui-daemon-host-name:8080/api/v1/supervisor?host=supervisor-daemon-host-name
2\. By id: http://ui-daemon-host-name:8080/api/v1/supervisor?id=f5449110-1daa-43e2-89e3-69917b16dec9-192.168.1.1
Request parameters:
| Parameter | Value | Description |
|---|---|---|
| id | String. Supervisor id | If specified, respond with the supervisor and worker stats with id. Note that when id is specified, the host argument is ignored. |
| host | String. Host name | If specified, respond with all supervisors and worker stats in the host (normally just one) |
| sys | String. Values 1 or 0. Default value 0 | Controls including sys stats part of the response |
Response fields:
| Field | Value | Description |
|---|---|---|
| supervisors | Array | Array of supervisor summaries |
| workers | Array | Array of worker summaries |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
Each supervisor is defined by:
| Field | Value | Description |
|---|---|---|
| id | String | Supervisor's id |
| host | String | Supervisor's host name |
| uptime | String | Shows how long the supervisor is running |
| uptimeSeconds | Integer | Shows how long the supervisor is running in seconds |
| slotsTotal | Integer | Total number of worker slots for this supervisor |
| slotsUsed | Integer | Number of worker slots used on this supervisor |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
| totalMem | Double | Total memory capacity on this supervisor |
| totalCpu | Double | Total CPU capacity on this supervisor |
| usedMem | Double | Used memory capacity on this supervisor |
| usedCpu | Double | Used CPU capacity on this supervisor |
Each worker is defined by:
| Field | Value | Description |
|---|---|---|
| supervisorId | String | Supervisor's id |
| host | String | Worker's host name |
| port | Integer | Worker's port |
| topologyId | String | Topology Id |
| topologyName | String | Topology Name |
| executorsTotal | Integer | Number of executors used by the topology in this worker |
| assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB) |
| assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB) |
| assignedCpu | Number | Assigned CPU by Scheduler (%) |
| componentNumTasks | Dictionary | Components -> # of executing tasks |
| uptime | String | Shows how long the worker is running |
| uptimeSeconds | Integer | Shows how long the worker is running in seconds |
| workerLogLink | String | Link to worker log viewer page |
Sample response:
{ "supervisors": [{ "totalMem": 4096.0, "host":"192.168.10.237", "id":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e-169.254.129.212", "uptime":"7m 8s", "totalCpu":400.0, "usedCpu":495.0, "usedMem":3432.0, "slotsUsed":2, "version":"0.10.1", "slotsTotal":4, "uptimeSeconds":428 }], "schedulerDisplayResource":true, "workers":[{ "topologyName":"ras", "topologyId":"ras-4-1460229987", "host":"192.168.10.237", "supervisorId":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e-169.254.129.212", "assignedMemOnHeap":704.0, "uptime":"2m 47s", "uptimeSeconds":167, "port":6707, "workerLogLink":"http:\/\/192.168.10.237:8000\/log?file=ras-4-1460229987%2F6707%2Fworker.log", "componentNumTasks": { "word":5 }, "executorsTotal":8, "assignedCpu":130.0, "assignedMemOffHeap":80.0 }, { "topologyName":"ras", "topologyId":"ras-4-1460229987", "host":"192.168.10.237", "supervisorId":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e-169.254.129.212", "assignedMemOnHeap":904.0, "uptime":"2m 53s", "port":6706, "workerLogLink":"http:\/\/192.168.10.237:8000\/log?file=ras-4-1460229987%2F6706%2Fworker.log", "componentNumTasks":{ "exclaim2":2, "exclaim1":3, "word":5 }, "executorsTotal":10, "uptimeSeconds":173, "assignedCpu":165.0, "assignedMemOffHeap":80.0 }] }
/api/v1/topology/summary (GET)
Returns summary information for all topologies.
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Topology Id |
| name | String | Topology Name |
| status | String | Topology Status |
| uptime | String | Shows how long the topology is running |
| uptimeSeconds | Integer | Shows how long the topology is running in seconds |
| tasksTotal | Integer | Total number of tasks for this topology |
| workersTotal | Integer | Number of workers used for this topology |
| executorsTotal | Integer | Number of executors used for this topology |
| replicationCount | Integer | Number of nimbus hosts on which this topology code is replicated |
| requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) |
| requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) |
| requestedTotalMem | Double | Requested Total Memory by User (MB) |
| requestedCpu | Double | Requested CPU by User (%) |
| assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB) |
| assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB) |
| assignedTotalMem | Double | Assigned Total Memory by Scheduler (MB) |
| assignedCpu | Double | Assigned CPU by Scheduler (%) |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
Sample response:
{ "topologies": [ { "id": "WordCount3-1-1402960825", "name": "WordCount3", "status": "ACTIVE", "uptime": "6m 5s", "uptimeSeconds": 365, "tasksTotal": 28, "workersTotal": 3, "executorsTotal": 28, "replicationCount": 1, "requestedMemOnHeap": 640, "requestedMemOffHeap": 128, "requestedTotalMem": 768, "requestedCpu": 80, "assignedMemOnHeap": 640, "assignedMemOffHeap": 128, "assignedTotalMem": 768, "assignedCpu": 80 } ], "schedulerDisplayResource": true }
/api/v1/topology-workers/:id (GET)
Returns the worker' information (host and port) for a topology.
Response fields:
| Field | Value | Description |
|---|---|---|
| hostPortList | List | Workers' information for a topology |
| name | Integer | Logviewer Port |
Sample response:
{ "hostPortList":[ { "host":"192.168.202.2", "port":6701 }, { "host":"192.168.202.2", "port":6702 }, { "host":"192.168.202.3", "port":6700 } ], "logviewerPort":8000 }
/api/v1/topology/:id (GET)
Returns topology information and statistics. Substitute id with topology id.
Request parameters:
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| window | String. Default value :all-time | Window duration for metrics in seconds |
| sys | String. Values 1 or 0. Default value 0 | Controls including sys stats part of the response |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Topology Id |
| name | String | Topology Name |
| uptime | String | How long the topology has been running |
| uptimeSeconds | Integer | How long the topology has been running in seconds |
| status | String | Current status of the topology, e.g. "ACTIVE" |
| tasksTotal | Integer | Total number of tasks for this topology |
| workersTotal | Integer | Number of workers used for this topology |
| executorsTotal | Integer | Number of executors used for this topology |
| msgTimeout | Integer | Number of seconds a tuple has before the spout considers it failed |
| windowHint | String | window param value in "hh mm ss" format. Default value is "All Time" |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
| replicationCount | Integer | Number of nimbus hosts on which this topology code is replicated |
| debug | Boolean | If debug is enabled for the topology |
| samplingPct | Double | Controls downsampling of events before they are sent to event log (percentage) |
| assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB) |
| assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB) |
| assignedTotalMem | Double | Assigned Off-Heap + On-Heap Memory by Scheduler(MB) |
| assignedCpu | Double | Assigned CPU by Scheduler(%) |
| requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) |
| requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) |
| requestedCpu | Double | Requested CPU by User (%) |
| topologyStats | Array | Array of all the topology related stats per time window |
| topologyStats.windowPretty | String | Duration passed in HH:MM:SS format |
| topologyStats.window | String | User requested time window for metrics |
| topologyStats.emitted | Long | Number of messages emitted in given window |
| topologyStats.trasferred | Long | Number messages transferred in given window |
| topologyStats.completeLatency | String (double value returned in String format) | Total latency for processing the message |
| topologyStats.acked | Long | Number of messages acked in given window |
| topologyStats.failed | Long | Number of messages failed in given window |
| workers | Array | Array of workers in topology |
| workers.supervisorId | String | Supervisor's id |
| workers.host | String | Worker's host name |
| workers.port | Integer | Worker's port |
| workers.topologyId | String | Topology Id |
| workers.topologyName | String | Topology Name |
| workers.executorsTotal | Integer | Number of executors used by the topology in this worker |
| workers.assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB) |
| workers.assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB) |
| workers.assignedCpu | Number | Assigned CPU by Scheduler (%) |
| workers.componentNumTasks | Dictionary | Components -> # of executing tasks |
| workers.uptime | String | Shows how long the worker is running |
| workers.uptimeSeconds | Integer | Shows how long the worker is running in seconds |
| workers.workerLogLink | String | Link to worker log viewer page |
| spouts | Array | Array of all the spout components in the topology |
| spouts.spoutId | String | Spout id |
| spouts.executors | Integer | Number of executors for the spout |
| spouts.emitted | Long | Number of messages emitted in given window |
| spouts.completeLatency | String (double value returned in String format) | Total latency for processing the message |
| spouts.transferred | Long | Total number of messages transferred in given window |
| spouts.tasks | Integer | Total number of tasks for the spout |
| spouts.lastError | String | Shows the last error happened in a spout |
| spouts.errorLapsedSecs | Integer | Number of seconds elapsed since that last error happened in a spout |
| spouts.errorWorkerLogLink | String | Link to the worker log that reported the exception |
| spouts.acked | Long | Number of messages acked |
| spouts.failed | Long | Number of messages failed |
| spouts.requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) |
| spouts.requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) |
| spouts.requestedCpu | Double | Requested CPU by User (%) |
| bolts | Array | Array of bolt components in the topology |
| bolts.boltId | String | Bolt id |
| bolts.capacity | String (double value returned in String format) | This value indicates number of messages executed * average execute latency / time window |
| bolts.processLatency | String (double value returned in String format) | Average time of the bolt to ack a message after it was received |
| bolts.executeLatency | String (double value returned in String format) | Average time to run the execute method of the bolt |
| bolts.executors | Integer | Number of executor tasks in the bolt component |
| bolts.tasks | Integer | Number of instances of bolt |
| bolts.acked | Long | Number of tuples acked by the bolt |
| bolts.failed | Long | Number of tuples failed by the bolt |
| bolts.lastError | String | Shows the last error occurred in the bolt |
| bolts.errorLapsedSecs | Integer | Number of seconds elapsed since that last error happened in a bolt |
| bolts.errorWorkerLogLink | String | Link to the worker log that reported the exception |
| bolts.emitted | Long | Number of tuples emitted |
| bolts.requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) |
| bolts.requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) |
| bolts.requestedCpu | Double | Requested CPU by User (%) |
Examples:
1\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825
2\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825?sys=1
3\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825?window=600
Sample response:
{ "name": "WordCount3", "id": "WordCount3-1-1402960825", "workersTotal": 3, "window": "600", "status": "ACTIVE", "tasksTotal": 28, "executorsTotal": 28, "uptime": "29m 19s", "uptimeSeconds": 1759, "msgTimeout": 30, "windowHint": "10m 0s", "schedulerDisplayResource": true, "topologyStats": [ { "windowPretty": "10m 0s", "window": "600", "emitted": 397960, "transferred": 213380, "completeLatency": "0.000", "acked": 213460, "failed": 0 }, { "windowPretty": "3h 0m 0s", "window": "10800", "emitted": 1190260, "transferred": 638260, "completeLatency": "0.000", "acked": 638280, "failed": 0 }, { "windowPretty": "1d 0h 0m 0s", "window": "86400", "emitted": 1190260, "transferred": 638260, "completeLatency": "0.000", "acked": 638280, "failed": 0 }, { "windowPretty": "All time", "window": ":all-time", "emitted": 1190260, "transferred": 638260, "completeLatency": "0.000", "acked": 638280, "failed": 0 } ], "workers":[{ "topologyName":"WordCount3", "topologyId":"WordCount3-1-1402960825", "host":"192.168.10.237", "supervisorId":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e-169.254.129.212", "uptime":"2m 47s", "uptimeSeconds":167, "port":6707, "workerLogLink":"http:\/\/192.168.10.237:8000\/log?file=WordCount3-1-1402960825%2F6707%2Fworker.log", "componentNumTasks": { "spout":5 }, "executorsTotal":8, "assignedMemOnHeap":704.0, "assignedCpu":130.0, "assignedMemOffHeap":80.0 }], "spouts": [ { "executors": 5, "emitted": 28880, "completeLatency": "0.000", "transferred": 28880, "acked": 0, "spoutId": "spout", "tasks": 5, "lastError": "", "errorLapsedSecs": null, "failed": 0 } ], "bolts": [ { "executors": 12, "emitted": 184580, "transferred": 0, "acked": 184640, "executeLatency": "0.048", "tasks": 12, "executed": 184620, "processLatency": "0.043", "boltId": "count", "lastError": "", "errorLapsedSecs": null, "capacity": "0.003", "failed": 0 }, { "executors": 8, "emitted": 184500, "transferred": 184500, "acked": 28820, "executeLatency": "0.024", "tasks": 8, "executed": 28780, "processLatency": "2.112", "boltId": "split", "lastError": "", "errorLapsedSecs": null, "capacity": "0.000", "failed": 0 } ], "configuration": { "storm.id": "WordCount3-1-1402960825", "dev.zookeeper.path": "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs": null, "topology.builtin.metrics.bucket.size.secs": 60, "topology.fall.back.on.java.serialization": true, "topology.max.error.report.per.interval": 5, "zmq.linger.millis": 5000, "topology.skip.missing.kryo.registrations": false, "storm.messaging.netty.client_worker_threads": 1, "ui.childopts": "-Xmx768m", "storm.zookeeper.session.timeout": 20000, "nimbus.reassign": true, "topology.trident.batch.emit.interval.millis": 500, "storm.messaging.netty.flush.check.interval.ms": 10, "nimbus.monitor.freq.secs": 10, "logviewer.childopts": "-Xmx128m", "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size": 1024, "storm.local.dir": "storm-local", "storm.messaging.netty.buffer_size": 5242880, "supervisor.worker.start.timeout.secs": 120, "topology.enable.message.timeouts": true, "nimbus.cleanup.inbox.freq.secs": 600, "nimbus.inbox.jar.expiration.secs": 3600, "drpc.worker.threads": 64, "topology.worker.shared.thread.pool.size": 4, "nimbus.seeds": [ "hw10843.local" ], "storm.messaging.netty.min_wait_ms": 100, "storm.zookeeper.port": 2181, "transactional.zookeeper.port": null, "topology.executor.receive.buffer.size": 1024, "transactional.zookeeper.servers": null, "storm.zookeeper.root": "/storm", "storm.zookeeper.retry.intervalceiling.millis": 30000, "supervisor.enable": true, "storm.messaging.netty.server_worker_threads": 1 }, "replicationCount": 1 }
/api/v1/topology/:id/metrics
Returns detailed metrics for topology. It shows metrics per component, which are aggregated by stream.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| window | String. Default value :all-time | window duration for metrics in seconds |
| sys | String. Values 1 or 0. Default value 0 | Controls including sys stats part of the response |
Response fields:
| Field | Value | Description |
|---|---|---|
| window | String. Default value ":all-time" | window duration for metrics in seconds |
| windowHint | String | |
| spouts | Array | Array of all the spout components in the topology |
| spouts.id | String | Spout id |
| spouts.emitted | Array | Array of all the output streams this spout emits messages |
| spouts.emitted.stream_id | String | Stream id for this stream |
| spouts.emitted.value | Long | Number of messages emitted in given window |
| spouts.transferred | Array | Array of all the output streams this spout transfers messages |
| spouts.transferred.stream_id | String | Stream id for this stream |
| spouts.transferred.value | Long | Number messages transferred in given window |
| spouts.acked | Array | Array of all the output streams this spout receives ack of messages |
| spouts.acked.stream_id | String | Stream id for this stream |
| spouts.acked.value | Long | Number of messages acked in given window |
| spouts.failed | Array | Array of all the output streams this spout receives fail of messages |
| spouts.failed.stream_id | String | Stream id for this stream |
| spouts.failed.value | Long | Number of messages failed in given window |
| spouts.complete_ms_avg | Array | Array of all the output streams this spout receives ack of messages |
| spouts.complete_ms_avg.stream_id | String | Stream id for this stream |
| spouts.complete_ms_avg.value | String (double value returned in String format) | Total latency for processing the message |
| bolts | Array | Array of all the bolt components in the topology |
| bolts.id | String | Bolt id |
| bolts.emitted | Array | Array of all the output streams this bolt emits messages |
| bolts.emitted.stream_id | String | Stream id for this stream |
| bolts.emitted.value | Long | Number of messages emitted in given window |
| bolts.transferred | Array | Array of all the output streams this bolt transfers messages |
| bolts.transferred.stream_id | String | Stream id for this stream |
| bolts.transferred.value | Long | Number messages transferred in given window |
| bolts.acked | Array | Array of all the input streams this bolt acknowledges of messages |
| bolts.acked.component_id | String | Component id for this stream |
| bolts.acked.stream_id | String | Stream id for this stream |
| bolts.acked.value | Long | Number of messages acked in given window |
| bolts.failed | Array | Array of all the input streams this bolt receives fail of messages |
| bolts.failed.component_id | String | Component id for this stream |
| bolts.failed.stream_id | String | Stream id for this stream |
| bolts.failed.value | Long | Number of messages failed in given window |
| bolts.process_ms_avg | Array | Array of all the input streams this spout acks messages |
| bolts.process_ms_avg.component_id | String | Component id for this stream |
| bolts.process_ms_avg.stream_id | String | Stream id for this stream |
| bolts.process_ms_avg.value | String (double value returned in String format) | Average time of the bolt to ack a message after it was received |
| bolts.executed | Array | Array of all the input streams this bolt executes messages |
| bolts.executed.component_id | String | Component id for this stream |
| bolts.executed.stream_id | String | Stream id for this stream |
| bolts.executed.value | Long | Number of messages executed in given window |
| bolts.executed_ms_avg | Array | Array of all the output streams this spout receives ack of messages |
| bolts.executed_ms_avg.component_id | String | Component id for this stream |
| bolts.executed_ms_avg.stream_id | String | Stream id for this stream |
| bolts.executed_ms_avg.value | String (double value returned in String format) | Average time to run the execute method of the bolt |
Examples:
1\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/metrics
1\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/metrics?sys=1
2\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/metrics?window=600
Sample response:
{ "window":":all-time", "window-hint":"All time", "spouts":[ { "id":"spout", "emitted":[ { "stream_id":"__metrics", "value":20 }, { "stream_id":"default", "value":17350280 }, { "stream_id":"__ack_init", "value":17328160 }, { "stream_id":"__system", "value":20 } ], "transferred":[ { "stream_id":"__metrics", "value":20 }, { "stream_id":"default", "value":17350280 }, { "stream_id":"__ack_init", "value":17328160 }, { "stream_id":"__system", "value":0 } ], "acked":[ { "stream_id":"default", "value":17339180 } ], "failed":[ ], "complete_ms_avg":[ { "stream_id":"default", "value":"920.497" } ] } ], "bolts":[ { "id":"count", "emitted":[ { "stream_id":"__metrics", "value":120 }, { "stream_id":"default", "value":190748180 }, { "stream_id":"__ack_ack", "value":190718100 }, { "stream_id":"__system", "value":20 } ], "transferred":[ { "stream_id":"__metrics", "value":120 }, { "stream_id":"default", "value":0 }, { "stream_id":"__ack_ack", "value":190718100 }, { "stream_id":"__system", "value":0 } ], "acked":[ { "component_id":"split", "stream_id":"default", "value":190733160 } ], "failed":[ ], "process_ms_avg":[ { "component_id":"split", "stream_id":"default", "value":"0.004" } ], "executed":[ { "component_id":"split", "stream_id":"default", "value":190733140 } ], "executed_ms_avg":[ { "component_id":"split", "stream_id":"default", "value":"0.005" } ] }, { "id":"split", "emitted":[ { "stream_id":"__metrics", "value":60 }, { "stream_id":"default", "value":190754740 }, { "stream_id":"__ack_ack", "value":17317580 }, { "stream_id":"__system", "value":20 } ], "transferred":[ { "stream_id":"__metrics", "value":60 }, { "stream_id":"default", "value":190754740 }, { "stream_id":"__ack_ack", "value":17317580 }, { "stream_id":"__system", "value":0 } ], "acked":[ { "component_id":"spout", "stream_id":"default", "value":17339180 } ], "failed":[ ], "process_ms_avg":[ { "component_id":"spout", "stream_id":"default", "value":"0.051" } ], "executed":[ { "component_id":"spout", "stream_id":"default", "value":17339240 } ], "executed_ms_avg":[ { "component_id":"spout", "stream_id":"default", "value":"0.052" } ] } ] }
/api/v1/topology/:id/component/:component (GET)
Returns detailed metrics and executor information
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| component | String (required) | Component Id |
| window | String. Default value :all-time | window duration for metrics in seconds |
| sys | String. Values 1 or 0. Default value 0 | controls including sys stats part of the response |
Response fields:
| Field | Value | Description |
|---|---|---|
| user | String | Topology owner |
| id | String | Component id |
| encodedId | String | URL encoded component id |
| name | String | Topology name |
| executors | Integer | Number of executor tasks in the component |
| tasks | Integer | Number of instances of component |
| requestedMemOnHeap | Double | Requested On-Heap Memory by User (MB) |
| requestedMemOffHeap | Double | Requested Off-Heap Memory by User (MB) |
| requestedCpu | Double | Requested CPU by User (%) |
| schedulerDisplayResource | Boolean | Whether to display scheduler resource information |
| topologyId | String | Topology id |
| topologyStatus | String | Topology status |
| encodedTopologyId | String | URL encoded topology id |
| window | String. Default value "All Time" | window duration for metrics in seconds |
| componentType | String | component type: SPOUT or BOLT |
| windowHint | String | window param value in "hh mm ss" format. Default value is "All Time" |
| debug | Boolean | If debug is enabled for the component |
| samplingPct | Double | Controls downsampling of events before they are sent to event log (percentage) |
| eventLogLink | String | URL viewer link to event log (debug mode) |
| profilingAndDebuggingCapable | Boolean | true if there is support for Profiling and Debugging Actions |
| profileActionEnabled | Boolean | true if worker profiling (Java Flight Recorder) is enabled |
| profilerActive | Array | Array of currently active Profiler Actions |
| componentErrors | Array of Errors | List of component errors |
| componentErrors.errorTime | Long | Timestamp when the exception occurred (Prior to 0.11.0, this field was named 'time'.) |
| componentErrors.errorHost | String | host name for the error |
| componentErrors.errorPort | String | port for the error |
| componentErrors.error | String | Shows the error happened in a component |
| componentErrors.errorLapsedSecs | Integer | Number of seconds elapsed since the error happened in a component |
| componentErrors.errorWorkerLogLink | String | Link to the worker log that reported the exception |
| spoutSummary | Array | (only for spouts) Array of component stats, one element per window. |
| spoutSummary.windowPretty | String | Duration passed in HH:MM:SS format |
| spoutSummary.window | String | window duration for metrics in seconds |
| spoutSummary.emitted | Long | Number of messages emitted in given window |
| spoutSummary.completeLatency | String (double value returned in String format) | Total latency for processing the message |
| spoutSummary.transferred | Long | Total number of messages transferred in given window |
| spoutSummary.acked | Long | Number of messages acked |
| spoutSummary.failed | Long | Number of messages failed |
| boltStats | Array | (only for bolts) Array of component stats, one element per window. |
| boltStats.windowPretty | String | Duration passed in HH:MM:SS format |
| boltStats.window | String | window duration for metrics in seconds |
| boltStats.transferred | Long | Total number of messages transferred in given window |
| boltStats.processLatency | String (double value returned in String format) | Average time of the bolt to ack a message after it was received |
| boltStats.acked | Long | Number of messages acked |
| boltStats.failed | Long | Number of messages failed |
| inputStats | Array | (only for bolts) Array of input stats |
| inputStats.component | String | Component id |
| inputStats.encodedComponentId | String | URL encoded component id |
| inputStats.executeLatency | Long | The average time a tuple spends in the execute method |
| inputStats.processLatency | Long | The average time it takes to ack a tuple after it is first received |
| inputStats.executed | Long | The number of incoming tuples processed |
| inputStats.acked | Long | Number of messages acked |
| inputStats.failed | Long | Number of messages failed |
| inputStats.stream | String | The name of the tuple stream given in the topology, or "default" if none specified |
| outputStats | Array | Array of output stats |
| outputStats.transferred | Long | Number of tuples emitted that sent to one ore more bolts |
| outputStats.emitted | Long | Number of tuples emitted |
| outputStats.stream | String | The name of the tuple stream given in the topology, or "default" if none specified |
Examples:
1\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout
2\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout?sys=1
3\. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout?window=600
Sample response:
{ "name": "WordCount3", "id": "spout", "componentType": "spout", "windowHint": "10m 0s", "executors": 5, "componentErrors":[{"errorTime": 1406006074000, "errorHost": "10.11.1.70", "errorPort": 6701, "errorWorkerLogLink": "http://10.11.1.7:8000/log?file=worker-6701.log", "errorLapsedSecs": 16, "error": "java.lang.RuntimeException: java.lang.StringIndexOutOfBoundsException: Some Error\n\tat org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)\n\tat org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)\n\tat org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)\n\tat backtype...more.." }], "topologyId": "WordCount3-1-1402960825", "tasks": 5, "window": "600", "profilerActive": [ { "host": "10.11.1.70", "port": "6701", "dumplink":"http:\/\/10.11.1.70:8000\/dumps\/ex-1-1452718803\/10.11.1.70%3A6701", "timestamp":"576328" } ], "profilingAndDebuggingCapable": true, "profileActionEnabled": true, "spoutSummary": [ { "windowPretty": "10m 0s", "window": "600", "emitted": 28500, "transferred": 28460, "completeLatency": "0.000", "acked": 0, "failed": 0 }, { "windowPretty": "3h 0m 0s", "window": "10800", "emitted": 127640, "transferred": 127440, "completeLatency": "0.000", "acked": 0, "failed": 0 }, { "windowPretty": "1d 0h 0m 0s", "window": "86400", "emitted": 127640, "transferred": 127440, "completeLatency": "0.000", "acked": 0, "failed": 0 }, { "windowPretty": "All time", "window": ":all-time", "emitted": 127640, "transferred": 127440, "completeLatency": "0.000", "acked": 0, "failed": 0 } ], "outputStats": [ { "stream": "__metrics", "emitted": 40, "transferred": 0, "completeLatency": "0", "acked": 0, "failed": 0 }, { "stream": "default", "emitted": 28460, "transferred": 28460, "completeLatency": "0", "acked": 0, "failed": 0 } ], "executorStats": [ { "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6701.log", "emitted": 5720, "port": 6701, "completeLatency": "0.000", "transferred": 5720, "host": "10.11.1.7", "acked": 0, "uptime": "43m 4s", "uptimeSeconds": 2584, "id": "[24-24]", "failed": 0 }, { "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6703.log", "emitted": 5700, "port": 6703, "completeLatency": "0.000", "transferred": 5700, "host": "10.11.1.7", "acked": 0, "uptime": "42m 57s", "uptimeSeconds": 2577, "id": "[25-25]", "failed": 0 }, { "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6702.log", "emitted": 5700, "port": 6702, "completeLatency": "0.000", "transferred": 5680, "host": "10.11.1.7", "acked": 0, "uptime": "42m 57s", "uptimeSeconds": 2577, "id": "[26-26]", "failed": 0 }, { "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6701.log", "emitted": 5700, "port": 6701, "completeLatency": "0.000", "transferred": 5680, "host": "10.11.1.7", "acked": 0, "uptime": "43m 4s", "uptimeSeconds": 2584, "id": "[27-27]", "failed": 0 }, { "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6703.log", "emitted": 5680, "port": 6703, "completeLatency": "0.000", "transferred": 5680, "host": "10.11.1.7", "acked": 0, "uptime": "42m 57s", "uptimeSeconds": 2577, "id": "[28-28]", "failed": 0 } ] }
Profiling and Debugging GET Operations
/api/v1/topology/:id/profiling/start/:host-port/:timeout (GET)
Request to start profiler on worker with timeout. Returns status and link to profiler artifacts for worker.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| host-port | String (required) | Worker Id |
| timeout | String (required) | Time out for profiler to stop in minutes |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Worker id |
| status | String | Response Status |
| timeout | String | Requested timeout |
| dumplink | String | Link to logviewer URL for worker profiler documents. |
Examples:
1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/start/10.11.1.7:6701/10
2\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/start/10.11.1.7:6701/5
3\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/start/10.11.1.7:6701/20
Sample response:
{ "status": "ok", "id": "10.11.1.7:6701", "timeout": "10", "dumplink": "http:\/\/10.11.1.7:8000\/dumps\/wordcount-1-1446614150\/10.11.1.7%3A6701" }
/api/v1/topology/:id/profiling/dumpprofile/:host-port (GET)
Request to dump profiler recording on worker. Returns status and worker id for the request.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| host-port | String (required) | Worker Id |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Worker id |
| status | String | Response Status |
Examples:
1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/dumpprofile/10.11.1.7:6701
Sample response:
{ "status": "ok", "id": "10.11.1.7:6701", }
/api/v1/topology/:id/profiling/stop/:host-port (GET)
Request to stop profiler on worker. Returns status and worker id for the request.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| host-port | String (required) | Worker Id |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Worker id |
| status | String | Response Status |
Examples:
1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/stop/10.11.1.7:6701
Sample response:
{ "status": "ok", "id": "10.11.1.7:6701", }
/api/v1/topology/:id/profiling/dumpjstack/:host-port (GET)
Request to dump jstack on worker. Returns status and worker id for the request.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| host-port | String (required) | Worker Id |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Worker id |
| status | String | Response Status |
Examples:
1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/dumpjstack/10.11.1.7:6701
Sample response:
{ "status": "ok", "id": "10.11.1.7:6701", }
/api/v1/topology/:id/profiling/dumpheap/:host-port (GET)
Request to dump heap (jmap) on worker. Returns status and worker id for the request.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| host-port | String (required) | Worker Id |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Worker id |
| status | String | Response Status |
Examples:
1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/dumpheap/10.11.1.7:6701
Sample response:
{ "status": "ok", "id": "10.11.1.7:6701", }
/api/v1/topology/:id/profiling/restartworker/:host-port (GET)
Request to request the worker. Returns status and worker id for the request.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| host-port | String (required) | Worker Id |
Response fields:
| Field | Value | Description |
|---|---|---|
| id | String | Worker id |
| status | String | Response Status |
Examples:
1\. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/restartworker/10.11.1.7:6701
Sample response:
{ "status": "ok", "id": "10.11.1.7:6701", }
POST Operations
/api/v1/topology/:id/activate (POST)
Activates a topology.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
Sample Response:
{"topologyOperation":"activate","topologyId":"wordcount-1-1420308665","status":"success"}
/api/v1/topology/:id/deactivate (POST)
Deactivates a topology.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
Sample Response:
{"topologyOperation":"deactivate","topologyId":"wordcount-1-1420308665","status":"success"}
/api/v1/topology/:id/rebalance/:wait-time (POST)
Rebalances a topology.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| wait-time | String (required) | Wait time before rebalance happens |
| rebalanceOptions | Json (optional) | topology rebalance options |
Sample rebalanceOptions json:
{"rebalanceOptions" : {"numWorkers" : 2, "executors" : {"spout" :4, "count" : 10}}, "callback" : "foo"}
Examples:
curl -i -b ~/cookiejar.txt -c ~/cookiejar.txt -X POST
-H "Content-Type: application/json"
-d '{"rebalanceOptions": {"numWorkers": 2, "executors": { "spout" : "5", "split": 7, "count": 5 }}, "callback":"foo"}'
http://localhost:8080/api/v1/topology/wordcount-1-1420308665/rebalance/0
Sample Response:
{"topologyOperation":"rebalance","topologyId":"wordcount-1-1420308665","status":"success"}
/api/v1/topology/:id/kill/:wait-time (POST)
Kills a topology.
| Parameter | Value | Description |
|---|---|---|
| id | String (required) | Topology Id |
| wait-time | String (required) | Wait time before rebalance happens |
Caution: Small wait times (0-5 seconds) may increase the probability of triggering the bug reported in STORM-112, which may result in broker Supervisor daemons.
Sample Response:
{"topologyOperation":"kill","topologyId":"wordcount-1-1420308665","status":"success"}
API errors
The API returns 500 HTTP status codes in case of any errors.
Sample response:
{ "error": "Internal Server Error", "errorMessage": "java.lang.NullPointerException\n\tat clojure.core$name.invoke(core.clj:1505)\n\tat org.apache.storm.ui.core$component_page.invoke(core.clj:752)\n\tat org.apache.storm.ui.core$fn__7766.invoke(core.clj:782)\n\tat compojure.core$make_route$fn__5755.invoke(core.clj:93)\n\tat compojure.core$if_route$fn__5743.invoke(core.clj:39)\n\tat compojure.core$if_method$fn__5736.invoke(core.clj:24)\n\tat compojure.core$routing$fn__5761.invoke(core.clj:106)\n\tat clojure.core$some.invoke(core.clj:2443)\n\tat compojure.core$routing.doInvoke(core.clj:106)\n\tat clojure.lang.RestFn.applyTo(RestFn.java:139)\n\tat clojure.core$apply.invoke(core.clj:619)\n\tat compojure.core$routes$fn__5765.invoke(core.clj:111)\n\tat ring.middleware.reload$wrap_reload$fn__6880.invoke(reload.clj:14)\n\tat org.apache.storm.ui.core$catch_errors$fn__7800.invoke(core.clj:836)\n\tat ring.middleware.keyword_params$wrap_keyword_params$fn__6319.invoke(keyword_params.clj:27)\n\tat ring.middleware.nested_params$wrap_nested_params$fn__6358.invoke(nested_params.clj:65)\n\tat ring.middleware.params$wrap_params$fn__6291.invoke(params.clj:55)\n\tat ring.middleware.multipart_params$wrap_multipart_params$fn__6386.invoke(multipart_params.clj:103)\n\tat ring.middleware.flash$wrap_flash$fn__6675.invoke(flash.clj:14)\n\tat ring.middleware.session$wrap_session$fn__6664.invoke(session.clj:43)\n\tat ring.middleware.cookies$wrap_cookies$fn__6595.invoke(cookies.clj:160)\n\tat ring.adapter.jetty$proxy_handler$fn__6112.invoke(jetty.clj:16)\n\tat ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown Source)\n\tat org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)\n\tat org.mortbay.jetty.Server.handle(Server.java:326)\n\tat org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)\n\tat org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)\n\tat org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)\n\tat org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)\n\tat org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)\n\tat org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)\n\tat org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)\n" }
