This document will define the API used by the various components of W.I.S.E. to communicate via MQTT.
All client applications participating in the MQTT communication must have their own unique ID. The unique ID should take the form <client type>_<process id>-<hostname>
but there is no requirement to use this format as long as the ID used is unique.
Client Types | Description |
---|---|
job | Information about a running W.I.S.E. job. |
wise | An instance of W.I.S.E. that is executing a job. |
builder | The W.I.S.E. job builder. |
manager | The W.I.S.E. job manager. |
delegator | A job delegator for load balancing. |
Some cross platform development environments have limited support for determining the running process ID and the hostname of the computer the process is running on. If a language is not able to find either of these values a random number will be used in their place to ensure that a unique client ID is still generated.
Client side apps should use a similar format but may need to replace the process ID and hostname fields with a value relevant to the platform.
All topics will have a similar structure for easy parsing.
{root path}/{originator}/{target}/{message type}
The root path will be a static string used for all communication. This document will assume the root path is always wise
. The originator
is the unique ID of the client that is publishing the message. The target can either be a specific client or a more general group of clients.
The message type is a unique identifier for the type of data being communicated in the payload as well as the intent of that data.
The topics that applications will respond to depends on whether they are the specific target in the topic, their application type is the target of the topic, or if the topic is a broadcast to all applications.
target
parameter will be the same value the client uses as the originator
when it publishes messages.target
. Ex. manager
can be used to communicate with all W.I.S.E. manager instances connected to the MQTT broker.target
. When the status of a running job changes it will publish messages for anyone who is watching the job to see by using the job name as the target
.broadcast
is used as the target
.There are several steps required to enable usage of MQTT.
-m
. Ex.java -jar WISE_builder.jar -m -j /mnt/shared/folder -l 32479
URL | Scheme | Description | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
emq.vm.sparcsonline.com:{port} | mqtt | Allows you to connect using the MQTT protocol. Both TCP and Websocket connections are available. |
||||||||||||
|
Asks clients to report that they are alive
All clients that match the client type will send a message to wise/{client id}/{originator}/checkin
when they receive this message. The message
from some clients may also include some diagnostic or status information. The payload is optional for this message.
client type
can be one of the following:
broadcast
- All client applications will respondwise
- Only W.I.S.E. will respondmanager
- Only W.I.S.E. Manager instances will respondbuilder
- Only W.I.S.E. Builder instances will respondName | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
qos | qos | integer | int32 | 1 | Quality of Service |
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
verbosity | integer | int32 | 1 | How verbose the return data should be. This has limited usage currently as most responses already contain little information. |
{
"verbosity": 1
}
A client was requested to checkin or W.I.S.E. Manager is changing state
Can either be a response to a wise/{target}/{client type}/reportin
request or from W.I.S.E. Manager changing state. When W.I.S.E. Manager is either starting or stopping it
reports that happening with this message so appropriate action can be taken.
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
qos | qos | integer | int32 | 1 | Quality of Service |
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
node_id | string | The ID the client uses when connecting to MQTT. | |||
version | string | The application version. | |||
status | integer | int32 | The applications status. | ||
node_type | integer | int32 | The type of application that is responding. | ||
topic_string | string | An optional value that repeats the base MQTT topic string that the application is listening to. | |||
manager_details | object | Diagnostic and status information specific to W.I.S.E. Manager. | |||
running_jobs | array (string) | A list of job IDs that are currently running in this W.I.S.E. Manager instance. Will not be returned with a requested verbosity of 0. | |||
memory_usage | integer | int64 | An optional value approximating the memory usage of the application in bytes. | ||
cpu_usage | number | double | The percent of the total available CPU (logical processors) used in a 1 second time period starting when the request was received. | ||
is_paused | boolean | Is the job queue paused so that no new jobs will be started. | |||
builder_details | object | Diagnostic and status information specific to W.I.S.E. Builder. | |||
memory_usage | integer | int64 | An optional value approximating the memory usage of the application in bytes. | ||
cpu_usage | number | double | The percent of the total available CPU (logical processors) used in a 1 second time period starting when the request was received. | ||
wise_details | object | Diagnostic and status information specific to W.I.S.E.. | |||
memory_usage | integer | int64 | An optional value approximating the memory usage of the application in bytes. | ||
cpu_usage | number | double | The percent of the total available CPU (logical processors) used in a 1 second time period starting when the request was received. |
{
"node_id": "manager_286588-COMPUTER",
"version": "7.2.6.1",
"status": 0,
"node_type": 1,
"topic_string": "wise",
"manager_details": {
"running_jobs": [
"job_20180221215557233"
],
"memory_usage": 1189171200,
"cpu_usage": 0
}
}
Reports the current status of a running job.
The W.I.S.E. executable will report the status of the currently executing job when various conditions are met. The originator
value will indicate which instance of W.I.S.E. is running the job.
The status may include any statistics outputs that were requested as part of the timestep settings when creating the job.
The status code may be one of the following values:
0
- The job has been submitted for execution.1
- W.I.S.E. has started running the job.2
- A scenario in the job has been started.3
- A scenario in the job has completed.4
- A scenario in the job has failed.5
- The job has completed.6
- The job has failed.7
- An unexpected error has occurred.8
- Generic execution information.9
- A request has been received to stop the job.10
- A file stream/upload request has completed.Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
qos | qos | integer | int32 | 1 | Quality of Service |
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
status | integer | int32 | Status Code | ||
message | string | A user readable message describing the status. | |||
stats | object | An optional collection of requested statistics emitted at the end of each timestep. All parameters in this object are optional depending on the job configuration. | |||
BURNED_AREA | number | double | Total area of the fire (sq. m). | ||
TOTAL_PERIMETER | number | double | Total fire perimeter (m), including interior and exterior and active/inactive portions. | ||
EXTERIOR_PERIMETER | number | double | Total exterior fire perimeter (m), including active and inactive portions. | ||
ACTIVE_PERIMETER | number | double | Portion of the fire front considered active (interior and exterior) (where 1 or both vertices are active) (m). | ||
TOTAL_PERIMETER_CHANGE | number | double | Change in the total perimeter growth (m). | ||
TOTAL_PERIMETER_GROWTH | number | double | Rate of change in the total perimeter growth rate (m/min). | ||
EXTERIOR_PERIMETER_CHANGE | number | double | Change in the exterior perimeter growth (m). | ||
EXTERIOR_PERIMETER_GROWTH | number | double | Rate of change in the exterior perimeter growth rate (m/min). | ||
ACTIVE_PERIMETER_CHANGE | number | double | Change in the active perimeter growth (m). | ||
ACTIVE_PERIMETER_GROWTH | number | double | Rate of change in the active perimeter growth rate (m/min). | ||
AREA_CHANGE | number | double | Change in fire area (sq. m). | ||
AREA_GROWTH | number | double | Rate of change in the fire area (sq. m). | ||
NUM_VERTICES | integer | int32 | Number of vertices defining the fire perimeter(s). | ||
NUM_ACTIVE_POINTS | integer | int32 | Number of active vertices defining the fire perimeter(s). | ||
CUMULATIVE_NUM_ACTIVE_POINTS | integer | int32 | Cumulative number of active vertices defining the fire perimeter(s). | ||
CUMULATIVE_NUM_POINTS | integer | int32 | Total, cumulative number of verticies defining the simulation's perimeters. | ||
NUM_FRONTS | integer | int32 | Number of fire fronts (interior and exterior). | ||
NUM_ACTIVE_FRONTS | integer | int32 | Number of fire fronts (interior and exterior) which have at least 1 active vertex. | ||
ROS | number | double | Maximum rate of spread (m/min). | ||
CFB | number | double | Maximum crown fraction burned (unitless). | ||
CFC | number | double | Maximum crown fuel consumption (kg/m2). | ||
SFC | number | double | Maximum surface fuel consumption (kg/m2). | ||
TFC | number | double | Maximum total fuel consumption (kg/m2). | ||
FI | number | double | Maximum fire intensity, based on ROS. | ||
FLAMELENGTH | number | double | Maximum flame length (m). | ||
TIMESTEP_REALTIME | integer | int64 | Number of real-time (clock) seconds to calculate the current display time step. | ||
TIMESTEP_CUMULATIVE_REALTIME | number | double | Number of real-time (clock) seconds to calculate all display time steps. | ||
TIMESTEP_CUMULATIVE_BURNING_SECS | number | double | Number of simulated seconds that burning was allowed since the start of the simulation. | ||
SCENARIO_CURRENT_TIME | string | The current simulation time (stored as ISO 8601). | |||
SCENARIO_NAME | string | The name of the scenario that the accompanying statistics are for. |
{
"status": 8,
"message": "The job status has changed",
"stats": {
"BURNED_AREA": 478528.421,
"SCENARIO_NAME": "scen0"
}
}
Cancel a job that has not yet started or terminate a job that is currently running.
The originator
value will be the ID of the process that sends the shutdown message.
The target
value must be the full ID of a running job.
There are three different priorities with which the job can be terminated.
0
- Shutdown when possible,1
- Shutdown when possible but attempt to forcibly close after a timeout (in minutes),2
- Forcibly close immediately.Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
qos | qos | integer | int32 | 1 | Quality of Service |
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
priority | integer | int32 | 0 | The urgency with which the system should attempt to stop the job from running. | |
timeout | integer | int32 | Only valid when priority is 1. This will be the number of minutes to wait for the job to shutdown on its own before attempting to terminate it. If not specified 5 minutes is used. |
{
"priority": 1,
"timeout": 5
}
Request the list of jobs that are known to a job manager.
The target
can either be a specific manager ID to only query a single instance or manager
to request that all managers connected to the same broker send their job list.
The returned jobs can be filtered by their current status within the manager:
all
- Return all jobs,run
- Only return running jobs,queue
- Only return jobs that are queued to be run,queuerun
- Both running and queued jobs will be returned,complete
- Only return jobs that have ended, either successfully or in error.If no payload is attached to the message or the filter is not one of the above values no filter will be applied to the response.
The managers that receive the message will respond by publishing a response message.
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
qos | qos | integer | int32 | 1 | Quality of Service |
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
filter | string | all | Filter the returned job list. | ||
offset | integer | Skip some job histories in the return values. This value is optional and if not present the entire job list will be returned. | |||
count | integer | Only return a specific number of job histories. This value is optional and if not present the entire job list will be returned. |
{
"filter": "all"
}
This is the response to the job list request message. The target
of this message will be the originator
of the request.
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
qos | qos | integer | int32 | 1 | Quality of Service |
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
total_count | integer | The total number of jobs that could be returned for the current filter. | |||
offset | integer | The offset into the possible return values that was requested. If no offset was requested this will be absent. | |||
count | integer | The maximum number of jobs to return. If no count was requested this will be absent. | |||
filter | string | The filter on the job list that was requested. | |||
jobs | array (object) | ||||
job_name | string | The name of the job. | |||
status | string | The current status of the job. Will be one of the filter values from the request message. | |||
submit_time | string | The time that the job was submitted at formatted as ISO-8601. May be absent for older jobs. | |||
start_time | string | The time that the job was started running at formatted in ISO-8601. Only present if the job has been started. | |||
complete_time | string | The time that the job completed at formatted in ISO-8601. Only present if the job completed. |
{
"total_count": 1,
"jobs": [
{
"job_name": "job_20180221215557233",
"status": "queue",
"submit_time": "2020-01-08T12:00:00-06:00"
}
]
}
System management requests. Not all client types are guaranteed to respect all types of management calls.
Possible request types are:
shutdown
- Shutdown the specified client or group of clients.reboot
- Restart the specified client or group of clients. W.I.S.E. Builder will not respect this request.pause
- Pause the job queue so that no new jobs will be started.unpause
- Unpause the job queue so that jobs can be started.offline
- Pauses the job queue and stops W.I.S.E. Manager from responding to requests for new jobs. Can only be undone by restarting W.I.S.E. Manager.zip
- Package all files from a job into a zip archive.tar
- Package all files from a job into a tar archive.delete
- Delete all files from a job.rerun
- Rerun a completed job. Use a manager ID in the topic to only start it for a single instance, otherwise use manager
for all manager instances that have the job to rerun it.Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
qos | qos | integer | int32 | 1 | Quality of Service |
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
request | string | The type of management being requested. | |||
target | string | The name of the job to act on, if needed for the operation. Only zip, tar, delete, and rerun require this value. | |||
delete_old | boolean | true | Should old job outputs be deleted before rerunning a job. Only rerun will use this value. If absent true is assumed. |
{
"request": "delete",
"target": "job_201812021421895"
}
Update the details of an already running job. Currently only cancelling scenarios is supported.
The payload can either be a single object to update or an array of objects.
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
qos | qos | integer | int32 | 1 | Quality of Service |
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
name | string | The name of the scenario that is to be editted. | |||
status | integer | int32 | Set to 0 or leave unspecified to not modify the status of the scenario. Set to 1 to cancel the scenario (will stop executing when possible if running or not be run if it has not yet started). |
[
{
"name": "scenario 1",
"status": 1
}
]
Published by W.I.S.E. Manager when a job has output a file that the creator has requested by streamed over MQTT. The target will always be the job that output the file.
Subscribers should respond with a /fileresponse topic if they want to receive the file in question. Manager will respond with another filerequest message that contains file data.
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
qos | qos | integer | int32 | 1 | Quality of Service |
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
job | string | The unique ID/name of the job that output the file. | |||
filename | string | A relative path and filename of the output file within the job directory. | |||
file_size | integer | int64 | The size of the file in bytes. | ||
is_end_of_file | boolean | Indicates if this is the last chunk of the file. If false another /fileresponse message should be sent with an incremented offset. Will only be present if an offset was present in the originating fileresponse message. | |||
data | string | A string wrapping UTF-8 encoded bytes read from the file. This is raw file data, not compressed or in any other way encoded. Will only be present if an offset was present in the originating fileresponse message. |
{
"job": "job_20180221215557233",
"filename": "scen0/outputfile.txt",
"is_end_of_file": false,
"file_size": 128,
"data": "F9D879978A98D98E"
}
This is the response message for /filerequest topics. The target should always be the ID of the W.I.S.E. Manager instance that was the originator or the /filerequest topic.
This topic initializes a partial transfer of the requested file from W.I.S.E. Manager to the requestor. The size of the returned chunk of the file is set by W.I.S.E. Manager.
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
qos | qos | integer | int32 | 1 | Quality of Service |
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
job | string | The unique ID/name of the job that output the file. | |||
filename | string | A relative path and filename of the output file within the job directory. | |||
offset | integer | int64 | An offset into the file to return. Although W.I.S.E. Manager sets the amount of data that can be returned in a single call the requestor must manage the offset when the file won't fit in a single payload. The offset should be increased by the size of the data chunk in the /filerequest topic. |
[
{
"job": "job_20180221215557233",
"filename": "scen0/outputfile.txt",
"offset": 0
}
]
When load balancing is enabled at a W.I.S.E. level (splitting the scenarios between multiple instances of W.I.S.E.) this call will request a new scenario index to run. The W.I.S.E. instances will continue to request scenario indices until the returned value is greater than the number of scenarios that are available to be run.
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
qos | qos | integer | int32 | 1 | Quality of Service |
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
job | string | The unique ID/name of the job that the next scenario index to run is being requested for. |
{
"job": "job_20180221215557233"
}
A response to a request for a scenario index to run for load balancing. The calling process will either run the scenario with the returned index or exit if the returned value is greater than the number of scenarios to run or negative.
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
qos | qos | integer | int32 | 1 | Quality of Service |
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
job | string | The unique ID/name of the job that the scenario index is for. | |||
index | integer | The index of the next scenario that should be run. |
{
"job": "job_20180221215557233",
"index": 1
}
If an FGM has been generated and requested to be validated by W.I.S.E. instead of immediately run this message will contain the results.
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
qos | qos | integer | int32 | 1 | Quality of Service |
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
success | boolean | Was the validation able to successfully run. This may be false if an older version of W.I.S.E. is used that doesn't support validation. | |||
valid | boolean | Is the FGM valid. | |||
load_warnings | string | User readable warnings generated when loading the FGM. Will be absent if there were no warnings. Warnings don't necessarily mean the FGM is not valid. | |||
error_list | object | A list of errors generated when loading the FGM. Will be absent if valid is true. |
{
"success": true,
"valid": true,
"load_warnings": "No load warnings have been generated."
}
Name | Title | Type | Format | Default | Description |
---|---|---|---|---|---|
MQTTQoSHeader | qos | integer | int32 | 1 | Quality of Service |
1