Giter VIP home page Giter VIP logo

tmo-dev's Introduction

tmo-dev

Dev and scratch space for tmo analysis routines dev.

Overview

Aims: develop some python classes for high(ish) level data analysis, including interactive plotting.

This mainly wraps or otherwise uses similar base functionality to that developed and demonstrated by the TMO team (thanks all!), plus adds some simple to use filtering, plotting and analysis functionality (using Xarray & Holoviews for the most part). It currently accepts preprocessed data only.

Demos are available in HTML format, with notebooks via docs/demos in this repo.

To many use of any of this, you'll need both the tmo-dev source code from Github, and also to set up your own Anaconda environment to install the additional packages required. General instructions are on the base class demo page and there's an Anaconda + package guide from SLAC here.

Releases

tmo-dev's People

Contributors

phockett avatar

Watchers

 avatar

tmo-dev's Issues

Electron VMI data and edge-effects

In LW06 tests (runs 81 - 87) seeing a lot of hits with only one "good" dimension, and the other at max - is this a binning issue, or "real" artifact (likely reflection current)? To look into - probably need to read more on the instrument first.

Pulling run info and configs (SACLA runs, plus general methods)

For SACLA workflow:

  • Pull run data from logfile (usually GSheet).
  • Additional data, e.g. ToFs and config settings usually posted in working channel (Slack, Discord).
  • Should configure for:
    • Config files per run/set of runs, and check for this when processing. (But may need to create manually.)
    • Labelling with additional info from logbook.

Basic logbook code example (from GSheet), working for 2023 runs:

# Source: https://stackoverflow.com/questions/19611729/getting-google-spreadsheet-csv-into-a-pandas-dataframe

# Tidy version
import pandas as pd

# Import Logbook
SHEET_ID = '<id number here>'
SHEET_NAME = 'Sheet1'
url = f'https://docs.google.com/spreadsheets/d/{SHEET_ID}/gviz/tq?tqx=out:csv&sheet={SHEET_NAME}'

logbook = pd.read_csv(url, header=0, index_col='Aq ')  # Get data from URL

# Clean up
logbook.index = logbook.index.astype(int)  # Force to int (otherwise float)
logbook.drop(logbook.columns[31:37], axis=1, inplace=True)  # Drop extra cols at the end
logbook.columns = logbook.columns.str.rstrip()   # Clean col name strings

# Display current
logbook.loc[runs]

Quick and dirty params setting in notebook via magic, e.g. %run -i 'aq189_params.py' where params file is just python vars from other sources.

Methods & plot types to add

  • General 1D & 2D histograms
  • Discrete histograms from another data axis
  • Raw data plots for testing/checking
  • Stacked plotting for multiple data types/dims
  • Polar plots & image ROI analysis
  • List data types & detectors at file IO

Filter methods rationalisation

Currently set for single filter set only in tmoDataBase class, but wrapped for multi-level filters in setFilter() method and VMI class. Should be unified architecture here (and back-propagate to early basic plotting functions), and more transparently applied.

May want a different branch for this, since it will likely break things.

Basic image analysis

Currently missing some basics for image analysis:

  • ROI setting & extraction.
  • ROI finding.
  • Polar coord conversion.
  • Circle finding.

Some of this already in psana...?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.