Home Intro Tutorials Articles Rest API Contact
Sign in Demo

Available parts

Step 4: Upload the csv files

By now, your data download is hopefully finished and it is time to upload the AIS data to the platform.

Click the Upload Data button in the top navigation bar:

The top navigation bar with the Upload data button clicked
Figure 1. The Upload Data button in the top navigation bar.

Step 4.1: Unzip the downloaded .zip file

The AIS data you downloaded comes in a .zip file. You cannot directly upload .zip files to the platform.

Use your favorite program to open .zip files and extract the AIS_2020_01_01.csv file from it and store it somewhere on your hard drive.

Step 4.2: Optionally, gzip the csv file

You can choose to upload the .csv file directly to the platform.

That file is over 700MB so uploading will take some time. To reduce this time, you can use gzip compression to first compress the file locally and then upload the compressed file.

  • On Windows, you can use 7-Zip to create gzip compressed files.

  • On Linux and macOS, you can use the gzip command on the command line

In case you downloaded multiple files and you want to compress them, you will have to gzip them one by one.

Step 4.3: Upload the file(s)

Now that you have your .csv files, or the compressed .csv.gz files, you can drag-and-drop them onto the data upload area.

Screenshot of the data upload area
Figure 2. The data upload area.

An alternative to drag-and-drop is clicking the area, which will show a file chooser where you can select your files.

You can drag-and-drop multiple files in one go

If you have multiple files to upload, you can select them all and drag-and-drop them in one go.

While your files are being uploaded, they will be listed in the Files Being Uploaded area.

Screenshot of the files being uploaded area
Figure 3. Data upload in progress.

Once the file is uploaded, it will appear in the Data Files area. Here you can see which of your files are already processed, which are still queued, and access the processing log for each file by clicking on the Status link.

Screenshot of the data files area
Figure 4. Once uploaded, the file appears in the CSV Files area.

The Processing Jobs area shows the different processing jobs that have ran for that data set. For example, if you would change the processing settings afterwards, a new job will be started, and you would see multiple entries here.

At first, a job will be queued until a processing node becomes available. Once the processing starts, the Job status will be updated.

Screenshot of the processing jobs area
Figure 5. The processing job, currently still in the Queued state
During upload, you can navigate to other pages in the xyzt.ai platform

Once you have drag-and-dropped the files you want to upload, you can safely navigate away from this page and continue using the platform while your upload finishes in the background.

The only requirement is that you keep the tab with the platform open to not stop the upload process.

Step 4.4: How to know if your data is ready?

When working with large or many .csv files, processing can take some time. Each uploaded file will be processed and at intermediate steps the results will be persisted to the cloud storage.

You can follow the status of the individual files as discussed above. When files have a status Uploaded or Processing, the data set is not ready. In addition, after handling the files, the persisting will take some time to finish.

You can follow the global process at the top of the page.

Screenshot of the incomplete processing progress
Figure 6. The processing progress as shown at the top of the data set creation pages. This data set is not finished and is still processing.
Screenshot of the finished processing progress
Figure 7. When processing is done, the checkmark (v) appears as well as an indication on the total number of records in the data set.

In addition, the processing progress is shown on the data set card:

  • Navigate to the Data sets page by clicking on Data sets in the navigation bar.

  • Click on YOUR DATA SETS (…​) and scroll to or search for the data set you are looking for.

  • The status of the data set is indicated in the lower left corner of the card:

    • Queued means that a processing job is queued for the data set, and the data set is not ready.

    • Processing means that data is still being processed, and the data set is not ready.

    • All data processed means that the data set is ready.

Screenshot of the processing jobs area
Figure 8. A data set that is not ready for analysis as it is still processing.

Next part

Go to the next part: Step 5: Use your data set