afm-spm / topostats-training Goto Github PK

Process images, gather dataset, get commands, make a few slides, and test myself.

We require a sample dataset for this tutorial so that there is a standard set of images for users to process as they work through the different stages.

I realised whilst working on #11 that we probably don't want to copy over the files and directory structure used in the Unix Shell Software Carpentry and would be better served to have a set of AFM images in a hierarchical structure which we can use to demonstrate directory structure and navigation in the Command Line sections and how TopoStats scans for images across the base_dir and how that in turn influences the structure of output.

For this I feel we need more than just minicircles.spm that is used in the TopoStats test suite.

It would also be worthwhile including an image that doesn't process so that we can demonstrate what happens when this arises and how to handle such cases.

If anyone has images that are not under Non-Disclosure Agreements and is able to share them please provide links in reply to this issue (or email me the Google Drive link, no need to send the image).

Develop an explanation of the configuration file and what the options are, what they could be and how to edit them.

• YAML is a recursive acronym and is a superset of the JavaScript Object Notation (JSON) file format.
• Text files that have keys and values and values can be nested keys.
• Text editor (not Word!) should be used for editing the files.

In addition to the documentation @SylviaWhittle has been developing a TopoStats User Guide which will need further bulking out with examples and explanations showing the impact of settings much in the vein already being used which shows the effects of...

• Aligining Rows
• Tilt removal
• Removal of polynomial trends
• Scar removal
• Zeroing the average height.

Much of this material can then be incorporated into this website for teaching and the TopoStats Documentation so we have examples available on a public facing website.

Target Audience

Assumed knowledge of Atomic Force Microscopy and the steps involved in processing images such as...

• Flattening
• Scar Removal
• Grain Finding
• Masks
• Tracing

Basic familiarity with computers is required but no familiarity with the Command Line Interface, or Conda environments is required, this is the content covered by the course. Python programming is not required as we are teaching users how to use TopoStats. Learning how to use the individual stages (as in existing notebooks) actually requires some Python basics and will, hopefully, be covered in future material that is developed.

Outcome Objectives

After undertaking the course attendees should have an understanding of and be able to perform the following.

Basic command line navigation

• Move to where images are (pwd / cd)
• List files/images (ls)
• Copying/move files (cp / mv).
• Understand how the command line relates to file manager views.

Conda basics (activate topostats env)

• Understand what an environment is
• Create environments
• Activate environments
• Install packages in the environment
• Install TopoStats
• Use terminal to install TopoStats via pip through conda

Configuration File Understand the Configuration File

• Know what each of the basic parameters do in the config file
• Process Images
• Run topostats on a folder of files
• Impact of parameters on images
• Understand what base_dir, output_dir do
• Understand what cores does
• Understand how filters effects images
• Understand how grains effects images
• Understand what core / all does

Stretch goal of plotting grainstats????

• Be able to use a topostats csv file to plot the area of molecules as a kde plot
• Possibly do this for two datasets?

Further to #9, both Linux and OSX are bundled with terminals/shells (Bash) that will work with the training material. It is better to try and preempt the problems that may arise from variation, to which end the Setup instructions need augmenting.

Windows

M$-Win however has a crude command-prompt with its own language.

In order to make life simpler and have one set of instructions that work across all operating systems we will have to include instructions for Windows users to install Git for Windows which includes a Bash shell. We can use the section described in the Software Carpentry : Unix Shell - Summary & Setup for Windows which links to Windows instructions.

OSX

We may wish to also link to instructions for Bash under OSX (see the OSX tab) as the default is ZSH (which will likely work, I use ZSH all the time under GNU/Linux and the instructions work).

Linux

Bash is the default on Linux, but users may use something else (personally I use ZSH, there is KSH, FISH and more). We should to link to instructions for Bash under Linux (see the Linux tab).

We would like to use some of the Conda environments for effective and reproducible research material as a basis for teaching the basics of the command line.

Sections we want to have covered, not necessarily in as much detail...

• Getting Started with Conda
  • What are the key terms I need to understand such as ‘package’, ‘dependency’ and ‘environment’?
  • Why should I use a package and environment management system as part of my research workflow?
  • What is Conda and why should I use it?
• Working with Environments
  • What is a Conda environment?
  • How do I create (delete) an environment?
  • How do I activate (deactivate) an environment?
  • How do I install packages into existing environments using Conda?
  • How do I find out what packages have been installed in an environment?
  • How do I find out what environments that exist on my machine?
  • How do I delete an environment that I no longer need?
• Using pip and Conda
  • What should I do if a Python package isn’t available via a Conda channel?
• Sharing Environments
  • Creating Environment Files using --from-history to limit what is captured.
  • Updating an Environment
  • Installing via Pip in environment.yaml

Need to make sure attribution is clearly given to comply with Creative Commons License under which the material is licensed.

We would like to use some of the Software Carpentries | UNIX Shell material as a basis for teaching the basics of the command line.

Sections we want to have covered, not necessarily in as much detail...

• Introducing the Shell
• Navigating Files and Directories
• Working with files

Need to make sure attribution is clearly given to comply with Creative Commons License under which the material is licensed.