This scenario will show how to upload a CSV data file containing batch information to Wizata. The process for uploading data is aligned with the method described in the article Upload a small data file through Azure Event Hub. Reviewing that article first will provide important context for this example.

🚧

Please use this logic for small data file, for full import use an import to the Time-Series DB directly.

📘

You can found this tutorial sample code on our GitHub as sample-03

Sample data file

As you may have already read from the previous article , we have prepared a sample file available on our GitHub. The dataset contains 24 hours of event data for multiple time-series batches. It will be used to illustrate event datapoint queries and Group System common use cases.

Structure

The csv file contains different key elements:

• A batch Id representing a batch number (a type of event)
• The name of the process. In this case, we will visualize the motor that was running at that particular time.
• Start and stop timestamps of the process.

Convert dataframe

You can find more information about all the different tags for Event business types datapoints here: Understanding Time-Series DB Structure: Events

Data sent should have this JSON structure:

{
    "Timestamp": "2024-11-13T00:00:00Z",
    "HardwareId": "mt1_bearings_tracking",
    "EventId": "H423F01",
    "EventStatus": "On"
}

{
    "Timestamp": "2024-11-13T01:48:31Z",
    "HardwareId": "mt1_bearings_tracking",
    "EventId": "H423F01",
    "EventStatus": "Off"
}

We should also send the "Off" event status corresponding to the stop time of the process.

To convert the dataframe given to the Wizata format, you can use a function like the one below:

def convert_dataframe(dataframe: pd.DataFrame) -> pd.DataFrame:
    data = []
    for index, row in dataframe.iterrows():
        data.append(
            {
                "Timestamp": pd.to_datetime(row.start, utc=True).isoformat(),
                "HardwareId": f"{row.process}_bearings_tracking",
                "EventId": row.batchId,
                "EventStatus": "On"
            }
        )
        data.append(
            {
                "Timestamp": pd.to_datetime(row.stop, utc=True).isoformat(),
                "HardwareId": f"{row.process}_bearings_tracking",
                "EventId": row.batchId,
                "EventStatus": "Off"
            }
        )
    return pd.DataFrame(data)

And then, the process to send the data via Event Hub is the same as the one described in the already mentioned article regarding small data file upload.