Giter VIP home page Giter VIP logo

powershellnotebook's Introduction

PowerShell Notebook

This module includes the function Invoke-PowerShellNotebook which enables you to run the cells inside the PowerShell notebook.

Continuous Integration Status


GitHub Actions
.github/workflows/ci.yml

Parameterizing a PowerShell Notebook

To parameterize your notebook designate a cell with the tag parameters. This is easy using Azure Data Studio.

Invoke-ExecuteNotebook looks for the parameters cell and treats this cell as defaults for the parameters passed in at execution time. Invoke-ExecuteNotebook will add a new cell tagged with injected-parameters with input parameters in order to overwrite the values in parameters. If no cell is tagged with parameters the injected cell will be inserted at the top of the notebook.

Executing a Notebook

You can execute the notebook in either a PowerShell script or at the command line.

Import-Module PowerShellNotebook

Invoke-ExecuteNotebook `
    -InputNotebook 'path/to/input.ipynb' `
    -OutputNotebook 'path/to/output.ipynb' `
    -Parameters @{ alpha=0.6; ratio=0.1 }

Execute via CLI

This executes the .\input.ipynb and outputs the results to the console.

Invoke-ExecuteNotebook .\input.ipynb
alpha = 1.2, ratio = 3.7, and alpha * ratio = 4.44

a = 5 and twice = 10

This executes the .\input.ipynb, injects the parameters, and outputs the results to the console.

Invoke-ExecuteNotebook .\input.ipynb -Parameters @{ alpha=0.6; ratio=0.1; a=10}
alpha = 0.6, ratio = 0.1, and alpha * ratio = 0.06

a = 10 and twice = 20

Save to a new notebook

You can execute the notebook and save it to a new notebook either locally, or as a a GitHub gist.

Note: You'll need to get a GitHub Public Access Token (PAT) and set $env:GITHUB_TOKEN to it.

Invoke-ExecuteNotebook .\input.ipynb gist://input_run1.ipynb -Parameters @{ alpha=0.6; ratio=0.1 }

More

Short video on Invoke-ExecuteNotebook

One of the uses of this Powershell tool is for parameterizing, and executing Jupyter Notebooks, Invoke-ExecuteNotebook. This opens up new opportunities for how notebooks can be used. For example you may have a financial report that you want to run with different values on the first or last day of a month using parameters makes this task easier.

Check out the Video Here

In a nutshell.

  1. Author your markdown with Chapter Start and End, then use fence blocks ``` to indic
  2. In Azure Data Studio PowerShell console, run Convert-MarkdownToNoteBook .\demo.md -watch
    • When you save the file, Convert-MarkdownToNoteBook detects and auto converts it to a PowerShell Notebook
  3. The converted Interactive PowerShell Notebook. Note: Convert-MarkdownToNoteBook also runs the code from the markdown file and includes the results.
  4. Thanks to Aaron Nelson and friends. Easily turn your SQL scripts into an interactive Jupyter Notebook with ConvertTo-SqlNoteBook. Plus, you can go the other way and turn an interactive Jupyter Notebook that has SQL in it, to a pure SQL script with Export-NotebookToSqlScript.

Convert a Markdown File to a PowerShell Notebook

Convert-MarkdownToNoteBook converts your markdown, recognizes fenced code blocks and converts it, and "runs" the PowerShell, including that snippets results in an executable document, a PowerShell notebook.

  • Share it with colleagues to better illustrate concepts
  • Develop workflows other can use, for diagnostics, testing, and more
Convert-MarkdownToNoteBook .\multiplePSLines.md

Exclude Results

Sometimes you don't want the results to be included in the converted PowerShell Notebook.

You can suppress evaluating the PowerShell code being run and the results being included by have the following comment as the first line in the fence block.

# Exclude Results

    ```ps
    # Exclude Results
    1+1
    ```

This will include the 1+1 in the interactive notebook, but will not evaluate the PowerShell statement and include it.

Available Functions

Function Synopsis
Add-NotebookCode Add-NotebookCode adds PowerShell code to a code block
Add-NotebookMarkdown Add-NotebookMarkdown adds Markdown to a markdown block
Add-NotebookMarkdown Add-NotebookMarkdown is intended to be used in a New-PSNotebook scriptblock
ConvertFrom-NotebookToMarkdown Take and exiting PowerShell Notebook and convert it to markdown
Convert-MarkdownToNoteBook Convert a markdown file to an interactive PowerShell Notebook
ConvertTo-SqlNoteBook Converts a .SQL file to a Notebook file
Export-NotebookToPowerShellScript Exports all code blocks from a PowerShell Notebook to a PowerShell script
Export-NotebookToSqlScript Exports all text & code blocks from a SQL Notebook to a .SQL file
Get-Notebook Get-Notebook reads the metadata of a single (or folder of) Jupyter Notebooks
Get-NotebookContent Get-NotebookContents reads the contents of a Jupyter Notebooks
Invoke-PowerShellNotebook Invoke-PowerShellNotebook executes all the PowerShell code blocks in a PowerShell Notebook.
New-PSNotebook Creates a new PowerShell Notebook that can be returned as text or saves as a ipynb file.
New-PSNotebookRunspace New-PSNotebookRunspace instantiates the PSNotebookRunspace
New-SqlNotebook Creates a new SQL Notebook that is saved as an .ipynb file

A PowerShell Notebook with Cells

Below is a PowerShell Notebook with three cells, each containing a PowerShell "script".

Notice the second cell has the results of running get-process | select company, name, handles -first 10

Automate the PowerShell Notebook

Bonus Points: Using the -AsExcel switch

Invoke-PowerShellNotebook sports an AsExcel switch. This lets you execute each cell in the PowerShell notebook and the function exports the results to a separate sheet in an Excel file.

You need to have the PowerShell ImportExcel module installed. The module is on the PowerShell Gallery, use Install-Module ImportExcel to install it on you machine.

A Little Language to Create PowerShell Notebooks

PowerShell Little Language

Create and Save a PowerShell Notebook

You can also create PowerShell Notebooks outside if Azure Data Studio with this module. Here is an example. It creates two code blocks and a markdown block, and saves it to a file C:\Temp\test.ipnyb.

New-PSNotebook -NoteBookName c:\temp\test.ipynb {
            Add-NotebookCode '$a=8'
            Add-NotebookCode '$a+12'
            Add-NotebookCode '$a+3'
            Add-NotebookMarkdown @'
## Math

- show addition
- show other
'@
}

Open the PowerShell Notebook in Azure Data Studio

You can open c:\temp\test.ipynb in Azure Data Studio and click Run Cells

image

Convert a demo.txt file to a PowerShell Notebook

If you've used start-demo.ps1 to setup PowerShell demos, this function will convert that format into a PowerShell Notebook.

ConvertTo-PowerShellNoteBook .\demo.txt .\demo.ipynb

Converts this to a PowerShell Notebook.

# Get first 10 process
ps | select -first 10

# Get first 10 services
gsv | select -first 10

# Create a function
function SayHello($p) {"Hello $p"}

# Use the function
SayHello World

Here it is in Azure Data Studio

SQL Notebooks

This module includes three functions for converting or creating SQL Notebooks which are compliant with the format used in Azure Data Studio.

For file conversion scenarios the ConvertTo-SqlNoteBook & Export-NotebookToSqlScript commands are available.

  1. ConvertTo-SqlNoteBook Will convert a .SQL file to a Notebook file (.ipynb)
  2. Export-NotebookToSqlScript Will export all text & code blocks from a SQL Notebook to a .SQL file
  3. The New-SqlNotebook Can creates a new SQL Notebook, and content can be added to that new SQL Notebook using the Add-NotebookMarkdown & Add-NotebookCode commands

Examples

The example below downloads the latest version of the BPCheck Jupyter Notebook from the TigerToolbox repository, converts it into a .SQL file (named BPCheck.SQL), and gets the content.

Export-NotebookToSqlScript "https://raw.githubusercontent.com/microsoft/tigertoolbox/master/BPCheck/BPCheck.ipynb"
Get-Content .\BPCheck.sql

The example below converts the AdventureWorksMultiStatementSBatch.sql file into a SQL Notebook.

ConvertTo-SQLNoteBook -InputFileName 'C:\temp\AdventureWorksMultiStatementSBatch.sql' -OutputNotebookName 'C:\temp\AdventureWorksMultiStatementSBatch.ipynb'

powershellnotebook's People

Contributors

dfinke avatar ili101 avatar jhoneill avatar markus-hanisch avatar sqlvariant avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

powershellnotebook's Issues

TODO - Enhance Invoke-ExecuteNotebook

  • Catch and report errors
  • Change new parameters to injected-parameters for New-CodeCell
  • If there are no cells tagged as parameters insert it at the beginning
  • Inject the new parameters only once, after the first
    - [ ] Handle targets like arrays passed in the hashtable
  • If the cell has not output.text, need to create it
  • Support reading a notebook from a URL

Issue with ConvertTo-PowerShell when there is <# and #>

Hello,

I tried converting a couple of PowerShell scripts from tigertoolbox this morning and found a few issues. The Notebook that I used to do the conversion is this: https://gist.githubusercontent.com/MsSQLGirl/b972d381fc85181aeafde5daa15ca171/raw/5d3db3ad14ef06304069bd74b53552b6879f15bf/PowerShellNotebook-ConversionIssue

Exception calling "Substring" with "2" argument(s): "Index and length must refer to a location within the string.
Parameter name: length"
At C:\Users\jukoesma\Documents\WindowsPowerShell\Modules\PowerShellNotebook\2.6.3\ConvertToPowerShellNoteBook.ps1:85 
char:28
+ ...  'Comment' {$TextBlock = $s.Substring($Block.Start, $Block.Length) -r ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ArgumentOutOfRangeException
 

I'm exhibiting this error whenever the source powershell file uses <# and #>

Update to advise min powershell version required

Received the following error.

At .....................\WindowsPowerShell\Modules\PowerShellNotebook\2.9.4\InvokeExecuteNotebook.ps1:61 char:100
+ ... rce -DotNetInteractive:$DotNetInteractive | ConvertFrom-Json -Depth 3
+                                                                  ~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [ConvertFrom-Json], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Microsoft.PowerShell.Commands.ConvertFromJsonCommand

Exception calling "Invoke" with "0" argument(s): "At line:27 char:38
+          throw "(╯°□°)╯︵ ┻�┻"
+                                      ~~~~~~~~
Unexpected token '»â”�â”»"
        exit
    }
}
<#
if ((Get-PackageProvider -Name NuGet).version -lt 2.8.5.201 ) {
    try {
        Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Confirm:$False -Force
    }
    catch [Exception]{
        $_.message                                                                                                                                                                                                                exit                                                                                                                                                                                                                  }                                                                                                                                                                                                                     }                                                                                                                                                                                                                         else {                                                                                                                                                                                                                        Write-Host "Version' in expression or statement.                                                                                                                                                                      At line:59 char:46
+ ... sion of NuGet installed = " (Get-PackageProvider -Name NuGet).version
+                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The string is missing the terminator: ".
At line:21 char:23
+     catch [Exception] {
+                       ~
Missing closing '}' in statement block or type definition.
At line:7 char:6
+ else {
+      ~
Missing closing '}' in statement block or type definition."
At .....................\Modules\PowerShellNotebook\2.9.4\PowerShellNotebookDSL.ps1:89 char:16
+         return $this.PowerShell.Invoke()
+                ~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ParseException

Found that the command ConvertFrom-Json does not use the parameter Depth on versions lower then 6.2, I was running on 5.1.

"Unexpected Token" errors importing PowerShellNotebook (version 2.9.3)

I was installing PowerShellNotebook from PSGallery without MinVersion/MaxVersion and recieved version 2.9.3

Get-Module PowerShellNotebook -ListAvailable


    Directory: C:\Program Files\WindowsPowerShell\Modules


ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Script     2.9.3      PowerShellNotebook                  {Add-NotebookCode, Add-NotebookMarkdown, Convert-MarkdownT...

When I tried to import the module I recieved the following:

> Install-Module PowerShellNotebook -Scope AllUsers -Repository PSGallery

> Import-Module PowerShellNotebook
At C:\Program Files\WindowsPowerShell\Modules\PowerShellNotebook\2.9.3\New-InteractiveNotebook.ps1:14 char:102
+ ... ls.dotnet-interactive-vscode/newNotebook?as={0}" -f ($AsDib ? 'dib':' ...
+                                                                 ~
Unexpected token '?' in expression or statement.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellNotebook\2.9.3\New-InteractiveNotebook.ps1:14 char:101
+ ... ools.dotnet-interactive-vscode/newNotebook?as={0}" -f ($AsDib ? 'dib' ...
+                                                                  ~
Missing closing ')' in expression.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellNotebook\2.9.3\New-InteractiveNotebook.ps1:1 char:34
+ function New-InteractiveNotebook {
+                                  ~
Missing closing '}' in statement block or type definition.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellNotebook\2.9.3\New-InteractiveNotebook.ps1:14 char:117
+ ... t-interactive-vscode/newNotebook?as={0}" -f ($AsDib ? 'dib':'ipynb'))
+                                                                        ~
Unexpected token ')' in expression or statement.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellNotebook\2.9.3\New-InteractiveNotebook.ps1:14 char:118
+ ... t-interactive-vscode/newNotebook?as={0}" -f ($AsDib ? 'dib':'ipynb'))
+                                                                         ~
Unexpected token ')' in expression or statement.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellNotebook\2.9.3\New-InteractiveNotebook.ps1:15 char:1
+ }
+ ~
Unexpected token '}' in expression or statement.
    + CategoryInfo          : ParserError: (:) [], ParseException
    + FullyQualifiedErrorId : UnexpectedToken

This was with Windows PowerShell 5.1

I've reverted to version 2.9.1 and was able to import PowerShellNotebook successfully.

Please let me know if I can provide any further information.

Handling of Get-NotebookContent conversion failures

It would be great if Get-NotebookContent could be a little more graceful when it runs into an error converting the JSON of a .ipynb file.

For example, this command brings back an error when it tries to process an empty file that just happens to have an .ipynb file extension.

dir -Filter *.ipynb | 
Get-Notebook -Path $_.FullName | 
Group KernelName

Instead of an error, a simple warning message might be better?

Finding all Jupyter Notebooks in subdirectories

When recursively searching through directories, I receive different results when I use foreach{...} around the Get-Notebook function, than when I pipe directly to that same command. If I leave off the -Recurse both versions of the command below return the same results. However, when I include the -Recurse the first version (the one without foreach{...}) only returns results from the top directory level, none of the results from lower directories are included.

dir -Filter *.ipynb -Recurse | 
Get-Notebook -Path $_.FullName | 
Group KernelName
dir -Filter *.ipynb -Recurse | %{
Get-Notebook -Path $_.FullName} | 
Group KernelName

Any support for .NET Interactive notebooks?

I notice that this was made to work with the PowerShell kernel that ships in Azure Data Studio. Do you know how well it works with the .NET Interactive kernel?

I'd love to be able to pull this in and use it there!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.