Coming soon on deephaven.io

Coming soon on deephaven.io

Coming soon on deephaven.io

Coming soon on deephaven.io

"a few code snippets that demonstrate crossing streams of data? Preferably, crossing (or intermingling) batch data with a real-time streams. We have this promotional idea to equate “crossing data streams” with crossing the streams from the Ghostbusters’ proton packs, since Halloween is right around the corner"

The iris.md file says that R.A. Fisher's paper came out in 1950. It actually came out in 1936.

Add README links to deephaven.io articles on loading data. This should be done after the CSV and Parquet interfaces have been reworked.

Users loading the GSOD sample data will see the ObservationDate field parsed as an inteegr. It should be a date.

Repro

1. Setup DH and run it with example data; you could do this, if you don't have it already:

mkdir fooey
cd fooey
curl https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/docker-compose.yml -O
docker-compose pull
docker-compose up

2. Connect to DH in your browser to get an IDE

3. Load the Tokyo observation data:

from deephaven.TableTools import readCsv

tokyo = readCsv("/data/examples/gsod/csv/tokyo.csv")

4. The ObservationDate column is an integer; not a date. The first two rows are 19,460,911 and 19,460,912. These are really September 11, 1946 and September 12, 1946.

The file is shaped like this:

USAF,WBAN,ObservationDate,TemperatureF,DewPointF,SeaLevelPressureMB,StationPressureMB,VisibilityFeet,MeanWindSpeedMPH,MaxWindSpeedMPH,MaxGustSpeedMPH,MaxTemperatureF,MinTemperatureF,PrecipitationInches
476710,99999,19460911,71.0,65.0,,,7.5,2.8,7.0,,74.3,68.4,0.0
476710,99999,19460912,73.9,61.5,,,11.2,6.8,13.0,,82.4,67.3,0.0
476710,99999,19460913,75.0,63.6,,,11.3,6.4,13.0,,80.2,71.2,0.0
476710,99999,19460914,73.4,66.4,,,10.5,6.6,16.9,,80.2,68.4,99.99
476710,99999,19460915,77.9,70.5,,,12.2,2.8,7.0,,84.4,73.4,0.0
476710,99999,19460916,72.6,63.4,,,11.6,4.0,13.0,,80.2,66.4,0.0
476710,99999,19460917,75.7,65.3,,,9.6,5.9,22.0,,81.3,69.3,99.99
476710,99999,19460918,72.8,70.0,,,6.0,5.2,14.0,,76.3,70.3,99.99

Expected

We should be able to load the sample data easily.

Or, maybe not: maybe on purpose we want to demonstrate how a common date format needs to be carefully manipulated and formatted at input as an exercise the learn-up the reader.

Add the deniro_pipe.csv file used by the first-csv-import how-to guide

As our DateTime library gets updated to use the Java Instant class instead, so should the example data contain actual time zones that users expect.

Coming soon on deephaven.io

CSV import tutorial uses this file, which is being renamed to "deniro.psv" from "deniro_pipe.csv". The file just needs to be renamed.

Add a readme that shows how to use the example repo.

Coming soon on deephaven.io

Actual Bones / No Bones readings for week 2 of November

Nov. 9 — Bones Day
Nov. 8 — Bones Day
Nov. 7 — No Bones Day
Nov. 6 — Bones Day

We should add a healthcheck script which verifies that all files have been downloaded and set that as the Dockerfile HEALTHCHECK. Without this, demo deployments can't tell if pulling examples failed or simply did not complete yet.

If the main download script touches a before/after marker file, we can make the HEALTHCHECK simply look for the after marker file to know that all data was successfully downloaded.

Coming soon on deephaven.io

Coming soon on deephaven.io