snowex-hackweek / website-2021 Goto Github PK
View Code? Open in Web Editor NEWJupyterBook for SnowEx Hackweek 2021
Home Page: https://snowex-2021.hackweek.io/
License: MIT License
JupyterBook for SnowEx Hackweek 2021
Home Page: https://snowex-2021.hackweek.io/
License: MIT License
tutorial documenting how to do this is here:
https://guides.github.com/activities/citable-code/
we want to make sure a release is made at the end of the hackweek so that all content contributors are listed
I'd like to demo sliderule-python in my tutorial. Can it be installed on the hub?
If Tutorial notebooks on the website read from S3:// resources that require authentication, we need to inject AWS CLI credentials into the continuous integration system. Otherwise when tutorial notebooks are executed we get authentication errors and can't retrieve data for subsequent computations and output
project pythia is currently working on a jupyter book that uses read-the-docs to 'preview' website changes before they get merged
ProjectPythia/projectpythia.github.io#17
our CI system is currently a little different. we run the build through docker so that we guarantee to use the exact same environment and package versions that are used on the hackweek jupyterhub.
i think we could use netlify to do website previews https://jupyterbook.org/publish/netlify.html
Figure directives are great for generating labels and having things look nice on the website:
```{figure} ../../img/sentinel1_radar_vision.jpg
---
height: 400px
name: sentinel1
---
Artist view of Sentinel-1. [Source](https://www.esa.int/ESA_Multimedia/Images/2014/01/Sentinel-1_radar_vision)
https://snowex-hackweek.github.io/website/tutorials/sar/sentinel1.html
But when actually running the notebook in a jupyterlab environment the figure is not rendered and just restates the directive:
---
height: 400px
name: sentinel1
---
Artist view of Sentinel-1. [Source](https://www.esa.int/ESA_Multimedia/Images/2014/01/Sentinel-1_radar_vision)
Seems the best solution is to use HTML tags as described here https://jupyterbook.org/content/figures.html#raw-html-images cc @mikedurand
We want to be able for GitHub Actions to run tutorial notebooks top-to-bottom to ensure that they run. Unfortunately, API-based access to NASA data (via curl
and wget
) is setup for using a ~/.netrc file rather than reading from environment variables or being able to specify a temporary token. This means we'll have to use a "machine user" with a NASA URS https://urs.earthdata.nasa.gov login and somehow inject those credentials into the continuous integration system.
For some reason, although only a few sentences of text only was changed on the last pull request, most of the plots on the UAVSAR tutorial are no longer rendering.
Just noting that some notebooks are not currently rendered with outputs on the website. These were merged pretty hastily and either never fully executed (error in a particular cell), or likely use a package that is missing from the default environment. Depending on how easy the fix is we can either leave them w/o outputs or try to fix.
WARNING: Execution Failed: /home/jovyan/book/tutorials/nsidc-access/nsidc-data-access.ipynb
WARNING: Execution Failed: /home/jovyan/book/tutorials/lidar/ASO_data_tutorial.ipynb
WARNING: Couldn't find cache key for notebook file book/tutorials/lidar/ASO_data_tutorial.ipynb. Outputs will not be inserted.
Last execution failed with traceback saved in /home/jovyan/book/_build/html/reports/ASO_data_tutorial.log
WARNING: Couldn't find cache key for notebook file book/tutorials/nsidc-access/nsidc-data-access.ipynb. Outputs will not be inserted.
Last execution failed with traceback saved in /home/jovyan/book/_build/html/reports/nsidc-data-access.log
/home/jovyan/book/tutorials/nsidc-access/nsidc-data-access.ipynb:10011: WARNING: Non-consecutive header level increase; 1 to 3 [myst.header]
/home/jovyan/book/tutorials/nsidc-access/nsidc-data-access.ipynb:330005: WARNING: Non-consecutive header level increase; 2 to 4 [myst.header]
During the hackweek it was very helpful to have tutorials organized by day, but now that the week is done people visiting the website either have to drill down to get to the right tutorial. https://snowex-hackweek.github.io/website/tutorials/index.html
An alternative to this organization is to create flashier landing page with an image and keywords for each tutorial, like we do for our team page: https://snowex-hackweek.github.io/website/team.html
Or a table with links like:
Tutorial | Topics | Datasets | Recording Link |
---|---|---|---|
Geospatial fundamentals | raster, vector, python | DEM, SNOTEL | Part 1, Part 2 |
cc @jomey
the preliminary work section of the website currently has a lot of material specific to icesat2 hackweek. just making an issue to keep track and will follow up with a PR to make things either generic or specific to snowex hackweek. https://snowex-hackweek.github.io/website/preliminary/index.html
described in these docs: https://jupyterbook.org/advanced/html.html#advanced-html-outputs
@dshean asked about the possibility of changing the website URL
We didn't put much thought into the website URL, but it's convenient to use GitHub pages hosting, which means the URL is fixed to the organization and repo name:
REPO -> WEBSITE
https://github.com/snowex-hackweek/website --> https://snowex-hackweek.github.io/website
A simpler option is maybe to map a custom domain name, so I think now that https://snowexdata.org
exists, @micahjohnson150 or @hpmarshall could link the current website to that URL ? which would be nice because then no changes need to happen here
to #15 (comment)
That said, it's pretty straightforward to change website
to whatever, such as jupyterbook
, but that would require updating a number of files and causing some confusion for participants.
For future events we can map the organizational page to the jupyterbook, but we thought it might be nice to someday have a flashy landing page that leads to the jupyterbook... https://snowex-hackweek.github.io
currently markdown links on several pages are not rendered correctly: https://snowex-hackweek.github.io/website/preliminary/jupyterhub.html#how-do-i-access-the-shared-cloud-environment shows up as Just click on [https://snowex.hackweek.io](https://snowex.hackweek.io)!
This is because of incorrectly using myst_substitutions with URLs as described here
https://jupyterbook.org/content/content-blocks.html?highlight=myst%20substitutions#using-substitutions-in-links
There are several ways to accomplish it, but for short URLs just using it verbatim is probably fine
UnboundLocalError Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_3204\3970605443.py in <cell line: 2>()
1 # call the tiff convert function, and it will print the information it extracted from the .ann file
----> 2 uavsar_tiff_convert(data_folder)
~\AppData\Local\Temp\ipykernel_3204\4031231228.py in uavsar_tiff_convert(folder)
106 sources.write(re.sub(line[8:13], Lines, line))
107 elif "samples" in line:
--> 108 sources.write(re.sub(line[10:15], Samples, line))
109 elif "LAT" in line:
110 sources.write(re.sub(line[12:15], Latitude, line))
UnboundLocalError: local variable 'Samples' referenced before assignment
https://github.com/snowex-hackweek/website/blob/main/book/norms/CoC.md#expectations
links to [email protected]
and the comment box is from IS2 hackweek. Should update these to be generic of have SnowEx 2021 titles.
Don created [email protected]
for our admin 'bot' user. Should we also have [email protected]
? cc @aaarendt @lsetiawan
People may be accustomed to formatting latex-style math equations in jupyter notebook markdown cells by copying and pasting equations from manuscripts
\begin{equation}
z = \frac{vt}{2} \quad
\end{equation}
This doesn't render correctly on the website by default, you have to enable the amsmath myst extension https://jupyterbook.org/content/math.html?highlight=equations#latex-style-math
Alternatively, using the following syntax works for simple equations
$$
z = \frac{vt}{2} \quad
$$ (equation_label)
We currently verify all links work with a GitHub Action workflow
https://github.com/snowex-hackweek/website/blob/main/.github/workflows/qaqc.yaml#L43-L57
Often there are failures having to do with checking certificates, such as (tutorials/geospatial/SNOTEL_query: line 20008) broken https://www.wcc.nrcs.usda.gov/snow/ - HTTPSConnectionPool(host='www.wcc.nrcs.usda.gov', port=443): Max retries exceeded with url: /snow/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)')))
Despite this error, the code or link does actually work.
@jomey pointed out that sphinx has the option to disable certificate verification here https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-tls_verify , so this could be set in the jupyterbook configuration file https://jupyterbook.org/customize/config.html
aws commands in a jupyter notebook cell like this !aws s3 sync s3://snowex-data/tutorial-data/camera-trap/ /tmp/camera-trap
currently work on jupyterhub but not when we build the website via CI. Instead of downloading data we get strange output: <botocore.awsrequest.AWSRequest object at 0x7f06fd62ae50>
#72 attempted a fix, but maybe we missed -e AWS_REGION
?
A good point raised by @dshean:
“how to cite,” but would be good to add. It’s a little more challenging with the tutorials as a subdirectory for an event-focused web page vs. a permanent archive, as the citation would be for the entire website repo.
Just attended an excellent presentation by @jules32 and @erinmr, which highlighted running notebooks hosted externally (that way it would be easy for each tutorial to have its own DOI) for this awesome NASA cookbook https://github.com/NASA-Openscapes/earthdata-cloud-cookbook . Wanted to note that alternative here for future events.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.