Context defines all properties, dataframes, models and other data passed through pipeline steps.
- properties are accessible on properties property.
- output dataframe(s) use return statement.
- input dataframe(s) get them by parameters or use dataframe property directly.
- for model use the set_model statement : support only one model per script.
- for plot use the set_model statement : support only one plot per script.
- dataframes, models and additional plots used within the pipelines can also be accessed, read-only.
Attributes
| Name | Type | Description |
|---|---|---|
| api | ApiInterface | api instance with limited functionality accessible during runner execution. |
| config | dict | all Wizata custom.ini configuration keys. |
| dataframe | pandas.Dataframe | single instance of pandas Dataframe inputted as parameter. |
| dataframes | dict | dict of all dataframes parameters and generated by previously executed steps. |
| datapoints | dict | wizata_dsapi.Datapoint for each name (key) used in queries. |
| execution_id | uuid.UUID | Execution ID currently processed. |
| grafana_api | grafana_client.GrafanaApi | instance of grafana api. |
| models | dict | dict of all models generated by previously executed steps. |
| now | datetime | Now timestamp as used by the pipeline. |
| pipeline_id | uuid.UUID | Pipeline ID associated with the ongoing execution. |
| plots | dict | dict of all plots generated by previously executed steps. |
| properties | dict | properties dictionary containing useful information such as variables. |
| step | PipelineStep | Currently processed step definition. |
| template | Template | Template if set on the pipeline. |
| registration | TwinRegistration | Registration associated with the pipeline and the twin currently processed. |
| warnings | list | all warnings and error messages. |
Methods
append()
append an object (pandas.Dataframe or any properties)
| Name | Type | Default | Description |
|---|---|---|---|
| key | str | dictionary identifier - name inside your pipeline. | |
| obj | ML Model, Dataframe or any properties (must be JSON serializable type). | ||
| overwrite | bool | True | by default - allow modifying an existing object. can be set to false. |
current_dataframes()
current dataframes a dictionary with all current named dataframes specific for this script.
dataframes contains all accessible dataframes for the pipeline mapped.
single dataframe context is accessible with context.dataframe and is not named
get()
get key from either dataframes, models, plots or properties.
| Name | Type | Default | Description |
|---|---|---|---|
| key | str |
return: None if not found.
get_model()
get model to be added to the context.
get_model_config()
extract model configuration from the context.
get_plot()
get plot set to be added to the context.
get_script_config()
extract script configuration from the context.
notify_execution()
notify the listeners and watchers on current execution status.
| Name | Type | Default | Description |
|---|---|---|---|
| execution | Execution |
notify_step()
notify the listeners and watchers on current step status.
| Name | Type | Default | Description |
|---|---|---|---|
| step_log | ExecutionStepLog |
reset()
reset context between step execution - remove all step info, but keep all data.
set_model()
set model to be added to the context.
| Name | Type | Default | Description |
|---|---|---|---|
| trained_model | Trained Model to be stored as a pickled object. | ||
| input_columns | |||
| output_columns | None | ||
| has_anomalies | False | ||
| scaler | None | Scaler to be stored if necessary. |
return: ML Model object prepared.
set_plot()
set plot to be added to the context.
| Name | Type | Default | Description |
|---|---|---|---|
| figure | Plotly figure. | ||
| name | Unkwown | Name of the plot. |
return: Plot object prepared.
write_log()
write log in console (only in experiment mode) - if running locally it will print in the console.
| Name | Type | Default | Description |
|---|---|---|---|
| message | str | message to write. | |
| level | int | 7 | from 7=DEBUG, 6=INFO, 3=ERROR to 1=CRITICAL |