Giter VIP home page Giter VIP logo

cpt-gef-txt-reader's Introduction

CPT-Parser (.gef.txt Files)

A simple pure-python implementation to parse .gef.txt files / CPT measurement data in ASCII format. Cone penetration tests (CPT) are a very common way of doing ground surveys.

The file format .gef.txt is based on the geotechnical exchange format (GEF) defintions and is very common to represent CPT test data. This repo shows a simple and efficient way of importing this data to a python dictionary/array. This array can easily imported as a pandas Dataframe or plotted with matplotlib.

1) Usage / Installation

  1. Copy the gef_parser.py file to your project
  2. Inside your python project import the parser function: from gef_parser import read_gef_file
  3. No dependencies or additional installs are needed to parse .gef.txt files

2) File Structure of a .gef.txt file

Header Information and beginning of the measurements of the CPT Test:

Projekt-Nummer: 12345678-12345
Projektname: Musterstadt, Drucksondierkampagne XYZ
Versuchs-Nummer: CPTU1
Kundenname: -
Ort: 12345 Musterstadt
Datum: 07.05.2023
Konus-Nummer: S22ABCDEF.S221234
Geländekante: 0,00
Wasserspiegel: 0,00
Vorbohrwerte: 0,00
E Coordinate: 0,000
N Coordinate: 0,000

Tiefe     qc        fs        u2        I         Rf        ic        Su_min    Su_max    soilfr    soilbq    soilavg   IFA       
          [MPa]     [MPa]     [MPa]     [°]       [%]                 [kPa]     [kPa]                                   [°]       

0,00      0,60      0,000     0,000     1,0       0,00      4,00      UNDEF     UNDEF     0,0       0,0       0,0       UNDEF     
0,01      0,60      0,000     0,000     1,4       0,00      4,00      UNDEF     UNDEF     0,0       0,0       0,0       UNDEF     
0,02      0,67      0,000     0,000     1,5       0,01      0,64      UNDEF     UNDEF     0,0       0,0       0,0       UNDEF     
0,03      0,75      0,000     0,000     1,4       0,02      0,46      UNDEF     UNDEF     0,0       0,0       0,0       UNDEF     
0,04      0,89      0,000     0,000     1,9       0,03      0,35      UNDEF     UNDEF     0,0       0,0       0,0       UNDEF     
0,05      1,04      0,001     0,000     1,7       0,06      0,36      UNDEF     UNDEF     0,0       0,0       0,0       UNDEF     

3) Examples to parse .gef.txt files

All code examples are located in ./example_usage.py as well.

Example 1: Parse a single file

from gef_parser import read_gef_file
# Example 1: Parse a single file
file_path = './sample_cptu_file.GEF.txt'
cpt_header, measurement_columns_and_units, measurements = read_gef_file(file_path)

print(measurements)
print(cpt_header)
print(measurement_columns_and_units)

Example 2: Create a pandas Dataframe from a single file

Parse a single file, create a pandas dataframe from the measurements and create a plot

import pandas as pd
import matplotlib.pyplot as plt

file_path = './sample_cptu_file.GEF.txt'
cpt_header, measurement_columns_and_units, measurements = read_gef_file(file_path)

# ## Create a pandas df from it and plot qc vs. depth
df = pd.DataFrame.from_dict(measurements)

# ## Data can easily be plotted
plt.plot(df['qc'], df['Tiefe'])
plt.gca().invert_yaxis()
plt.xlabel("qc [MPa]")
plt.ylabel("Depth [m]")
plt.show()

Example 3: Read multiple .gef.txt files in a folder

This example code parses all .gef.txt files in a folder and subfolders. Then it creates a big pandas data-frame of the measurement data and associated cpt header data. Finally it exports the data into a single Excel file. The two tables can easily imported into Leapfrog works for geological modelling.

# Parse all *.gef.txt files in the GEF_SAMPLES folder
from glob import glob
folder_path = './GEF_SAMPLES/**/*.GEF.txt'
all_measurements = []
all_cpt_header_info = []
for file_path in glob(folder_path):
    cpt_header, measurement_columns_and_units, measurements = read_gef_file(file_path)
    measurements = [dict(item, **{'aufschluss_name':cpt_header['aufschluss_name']}) for item in measurements] # add aufschluss_name to the measurements
    all_measurements += measurements
    all_cpt_header_info.append(cpt_header)

## optionally add them to a pd.Dataframe and export it to a single excel file
import pandas as pd
header_df = pd.DataFrame.from_dict(all_cpt_header_info)
measurements_df = pd.DataFrame.from_dict(all_measurements)
excel_outpath = './CPT_Summary.xlsx'

# depending on the ampount of cpt measurement data the excel file can take some time
with pd.ExcelWriter(excel_outpath) as writer:
    header_df.to_excel(writer, sheet_name="Sheet0_cpt_header_info", index=False)
    measurements_df.to_excel(writer, sheet_name="Sheet1_cpt_measurements", index=False)

# optionally export the data into two csv files that can be imported into Leapfrog Works
header_df.to_csv('collar.csv')
measurements_df.to_csv('intervals.csv')

Example 4: Read multiple .gef.txt files in a folder and create a thiessen-polygon and a delauny triangulation

This example code parses all .gef.txt files in a folder and subfolders. Then it creates a list of all input xyz coordinates and computes a delauny triangulation and thiessen/voronoi polygons. You can export the polygons into a shapefile/wkt-csv file to view it in QGIS etc.

# Example 4: Create Thiessen Polygons and a Delauny triangulation from CPT xyz-Coordinates
from glob import glob
folder_path = './GEF_SAMPLES/**/*.GEF.txt'
all_measurements = []
all_cpt_header_info = []
for file_path in glob(folder_path):
    cpt_header, measurement_columns_and_units, measurements = read_gef_file(file_path)
    measurements = [dict(item, **{'aufschluss_name':cpt_header['aufschluss_name']}) for item in measurements] # add aufschluss_name to the measurements
    all_measurements += measurements
    all_cpt_header_info.append(cpt_header)

# Create Delauny and Thiessen Polygons
default_points = [(0,0,0),(5,5,0),(5,0,0), (10,3,2)]
points = [(cpt.get('RW'), cpt.get('HW')) for cpt in all_cpt_header_info if(cpt.get('RW') is not None and cpt.get('HW') is not None)]
if(len(points)==0):
    points = default_points

triangle_polygons = pytess_triangulate(points)
print(triangle_polygons)

thiessen_polygons = pytess_voronoi(points, buffer_percent=100)
print(thiessen_polygons)

print("...do some stuff with the triangulation")

cpt-gef-txt-reader's People

Contributors

vmarquar avatar

Stargazers

 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.