This guide offers a reference for creating custom features in ArcGIS Insights using Python and R. It is the definitive guide for Insights scripting topics and a resource for implementing Jupyter's Kernel Gateway.
- ArcGIS Insights (version 2020.x)
- Anaconda (version 3.7)
- See needed Python and R dependencies
Note: Scripting is not supported in Insights running in ArcGIS Online. Please download Insights Desktop for this instead, which supports ArcGIS Online connections, ArcGIS Enterprise connections, database and scripting features.
You can access an archived version of this documentation here.
Insights supports connections to Jupyter's Kernel Gateway version 2.1.0, which is an open source web server distributed through conda-forge
and other repository channels. To setup a Kernel Gateway, with the required dependencies choose one of the following deployment sections.
Check out Deployment Patterns for system planning recommendations.
-
Install Anaconda v3.7
-
Create a folder named
gateway
-
Copy
selfsign.py
andinsights-base.yml
intogateway
folder -
Open Anaconda's command promt and CD into the
gateway
folder -
Run below commands
conda env create -f insights-base.yml conda activate insights-base python selfsign.py
-
Start the Kernel Gateway:
-
Run this command if using Insights in ArcGIS Enterprise
jupyter kernelgateway --KernelGatewayApp.ip=0.0.0.0 --KernelGatewayApp.port=9999 --KernelGatewayApp.allow_origin='*' --KernelGatewayApp.allow_credentials='*' --KernelGatewayApp.allow_headers='*' --KernelGatewayApp.allow_methods='*' --JupyterWebsocketPersonality.list_kernels=True --certfile=./server.crt --keyfile=./server.key
-
Run this command if using Insights Desktop
jupyter kernelgateway --KernelGatewayApp.ip=0.0.0.0 --KernelGatewayApp.port=9999 --KernelGatewayApp.allow_origin='*' --KernelGatewayApp.allow_credentials='*' --KernelGatewayApp.allow_headers='*' --KernelGatewayApp.allow_methods='*' --JupyterWebsocketPersonality.list_kernels=True
- Optional: Stop Kernel Gateway by pressing Control-C in the running window or close the window
-
Install Docker
-
Create a folder named
gateway
-
Copy
selfsign.py
andDockerfile
intogateway
folder -
Run
selfsign.py
to create certificates in thegateway
folderpython selfsign.py
-
Create a
data
folder withingateway
and put your data files there -
Run this command to create the Kernel Gateway Docker image
docker build -t insights-gateway .
-
Start the Kernel Gateway
docker run -p 9999:9999 insights-gateway
To create a connection to your Kernel Gateway follow these steps:
- Open Insights
- Create a new workbook
- Click the Scripting icon
- Complete Kernel Gateway connection form
Note: Connections must reference the Kernel Gateway root URL. For tips on what connections may look like see Connection examples.
Urls may be HTTP or HTTPS. Hosts can be referenced in numerous ways, IP address, localhost, FQDN etc. You can use any available inbound port number that is not already in use. If using 443, a connection will not require the port number. Here are some examples. Yes means connection schema is supported. No, means that URL connection will likely fail (not work).
Connection URL | Insights in Enterprise | Insights Desktop |
---|---|---|
http://localhost:9999 | no | yes |
https://localhost:9999 | no | no |
http://pickle:9999 | no | yes |
https://pickle:9999 | no | no |
http://12.120.95.153:9999 | no | yes |
https://12.120.95.153:9999 | yes | no |
http://pickle.esri.com:9999 | no | yes |
https://pickle.esri.com:9999 | yes | no 1 |
1 Insights Desktop can make connections to HTTPS Kernel Gateway endpoints, if the Kernel Gateway uses a domain or a certificate authority certificate.
2 If using port 443, the connection url will look like this https://pickle.esri.com
.
Python and R scripting features are distributed across the app. Shared scripts are accessed from the Add dialog. Script modules and module options are accessed via the Data Pane. Lastly, the Console itself has many script features. Refer to this table for an overview of tools and capabilities.
The console enables keyboard shortcuts to perform routine tasks quickly.
Shortcut | Description |
---|---|
Ctrl + B | Create comments for selected code. |
Shift + Enter | Executes code in current cell. |
Ctrl + Alt + B | Adds %insights_return(<data frame object>) magic command to cell |
The console supports the following magic command. This magic command must be placed in it's own cell.
Magic command | Description |
---|---|
%insights_return(<data frame object>) |
Converts Python or R data frames into Insights datasets. When %insights_return(df) is run it will generate an Insights dataset from the df object. Data will be persisted in the workbook (when the workbook is saved) and will appear in the data pane after execution. |
There are various configurations to choose from when planning a Jupyter Kernel Gateway with Insights. It should be noted that some configurations may have tactical advantages over others. Additionally, each configuration will offer different end user experiences and varying degrees of effort regarding setup and maintenance.
These conceptual diagrams were designed to help organizations visualize different kinds of Jupyter Kernel Gateway configurations next to different kinds of Insights deployments.
- This configuration entails low newtworking and firewall considerations
- Data files may live on personal computer or file server
- This configuration entails moderate networking and firewall considerations and skills
- Data files should live on file server or Kernel Gateway machine
- This configuration entails moderate networking and firewall considerations and skills
- Data files should live on file server or Kernel Gateway machine
- This configuration entails moderate networking and firewall considerations and skills
- Data files may live on personal computer or file server
- Data files may need to be accessible from the cloud
- This configuration entails advanced networking and firewall skills and considerations
Part of the Esri Geospatial Cloud, ArcGIS Insights is data analytics made for advanced location intelligence. Using Insights you can ask questions you did not know to ask, analyze data completely, and tell powerful data stories. Connect to your data directly, then use maps, charts, tables and reuseable models and scripts to perform basic to complex analyses that scale based on skill level and business need.
You can do this by putting an explanation point in front of a pip install command. Like,
!pip install BeautifulSoup4
If all goes well (after running the command), download activity will apear in the output cell. When the command finishes, you can then import your library and run scripts like normal.
from bs4 import BeautifulSoup
soup = BeautifulSoup("<p>Hello Insights!</p>")
print(soup.prettify())
Insights is running in the web browser and when making a Kernel Gateway connection it says "Not able to add this connection. Try with a different URL or web socket or check if your gateway is running."
If you've followed the guide (and ran the selfsign.py file), you have created a self signed SSL certificate. It may be possible that Insights cannot make a connection because the web browser itself does not trust the certificate. To work around this problem open the kernel gateway URL in the web browser and accept the browser warning. Then try connecting again.
My Kernel Gateway is on a different machine and I am having trouble making a connection using Insights?
A fundamental way to troubleshoot this problem is confirm that all needed computers can talk to each other. If you are running Insights in Enterprise this means each ArcGIS Server machine, plus your Kernel Gateway and personal computer must all be able to communicate with each other. Insights Desktop entails less troubleshooting. For Insights Desktop deployments, only the Kernel Gateway and your personal computer need to talk to each other.
Try getting the IP address of:
- Your personal computer machine
- Your kernel gateway machine
- Your ArcGIS Server machine(s)
and then from each machine run the ping
command to see if ping messages are received.
Tip: On windows, run ipconfig
and reference the Iv4 address to get the IP address. On mac, run ipconfig getifaddr en0
and note the address.
Copyright 2020 Esri
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
A copy of the license is available in the repository's license.txt file.