ubc-mds / 2023-dsci522-group22 Goto Github PK
View Code? Open in Web Editor NEWUBC MDS DSCI-522 Group Project
Home Page: https://ubc-mds.github.io/2023-DSCI522-Group22/wine_color_classification_report.html
License: Other
UBC MDS DSCI-522 Group Project
Home Page: https://ubc-mds.github.io/2023-DSCI522-Group22/wine_color_classification_report.html
License: Other
Using the build_jupyter_book branch.
Here are some tips:
You'll have to conda install jupyter-book=0.15.1 or update from the yaml file.
https://jupyterbook.org/en/stable/basics/build.html <- how to build
The model section still needs to be done.
I couldn't figure out how to get a self contained html file built from our base /reports directory similar to https://github.com/ttimbers/breast_cancer_predictor_py/tree/main/report
This is mostly directed to chun li.
When you are finished your script let me know , so I can see if I need to add more function to my script? I made an educated guess everything my script was supposed to include, but not 100% confident.
python scripts/wine_classification_plot_script.py
--dataframe-csv results/tables/logistic_regression_feature_importance.csv
--red-wine-color red
--white-wine-color peachpuff
--output-path results/figures/predict_visualization.png
This is how to execute my script.
Not sure if we are storing this somewhere so we can update the readme.
Hello team, I am working on converting a visualization to a function. But I wonder how to test it. It seems that it is not possible to test if a chart is bar chart or scatter plot. Any ideas?
When I run the script, it gave me the following error, please check it:
<img width="1173" alt="image" src="https://github.com/UBC-MDS/2023-DSCI522-Group22/assets/108684429/f21b7748-6ef6-40b8-80e9-1f60a64775d5">
I forgot to update the docker file and yml file with new package: "click". Could someone approve my pull request and then perhaps add that? Thank you in advance :)
I sent a pull request for the combined data file. Please check it.
pre-emptively anticipating feedback, on Monday I got our github pages to render our report.
Today and tomorrow I'm dividng my attention between 573 and 522. I'll create the changelog and get onto fixing some of the suggestions.
I'm a little confused why group feedback isn't in our issues? I must have misinterpreted the instructions about the location.
I'll figure it out in 522 lecture today, I'll ask Tiff/TA.
I've sent a pull request for the makefile. Please test it.
Hi team
I'm happy to report that our report is now fully reproducible on Docker with the instructions from our README.md file. Here's what I did:
Also, test_download_failure_invalid_url function failed too.
`__________________________________________ test_download_failure_invalid_url __________________________________________
def test_download_failure_invalid_url():
runner = CliRunner()
result = runner.invoke(main, ['--url', 'invalidurl://example.com/somefile.csv'])
assert result.exit_code != 0
E assert 0 != 0
E + where 0 = .exit_code
test_download_file.py:28: AssertionError
`
No idea to fix it.
Ok, I just pull requested the tests/functions for my part. This will complete the modularization part of our project.
Tomorrow we need to double check things (verify .yaml versions are correct , everyone's tests pass)
, make sure we've published the container image on DockerHub and publish the final report from the readme.
Good work team!
My test function gives the following error message:
`______________________________________________ test_download_http_error _______________________________________________
tmp_path = PosixPath('/private/var/folders/c3/4qxt4f9j08z9s0bydw9k0n800000gn/T/pytest-of-f1yingh3ap/pytest-6/test_download_http_error0')
def test_download_http_error(tmp_path):
test_url = "https://opendata.vancouver.ca/api/explore/v2.1/catalog/datasets/issued-building-permits/exports/csv?lang=en&timezone=America%2FLos_Angeles&use_labels=true&delimiter=%3B"
test_destination = tmp_path.as_posix()
mock_response = Mock()
mock_response.raise_for_status.side_effect = requests.exceptions.HTTPError
with patch('download_file.requests.get', return_value=mock_response):
runner = CliRunner()
result = runner.invoke(main, ['--url', test_url, '--destination', test_destination])
assert result.exit_code != 0
E assert 0 != 0
E + where 0 = .exit_code
test_download_file.py:41: AssertionError
=============================================== short test summary info ===============================================
FAILED test_download_file.py::test_download_failure_invalid_url - assert 0 != 0
FAILED test_download_file.py::test_download_http_error - assert 0 != 0
============================================= 2 failed, 1 passed in 0.12s =============================================
`
Any idea to fix it?
Based on our prior agreement, in order to allow each member to have a chance to add a few lines to the Docker File, we will have to make 4 branches and 4 Pull Requests, which seems tedious based on something that is trivial. How about we just make the changes on the Main branch instead only for this task?
Hey all, wanted to let you know that I'm working on removing some unnecessary EDA steps from our Jupyter book that won't be useful in communicating our project thesis to a general audience. Ill make a sub branch and push it for review when I'm done.
Here's a non-exhaustive to do list for this week's group project, based on the differences between our current repo and the sample repo from Tiffany.
Hey guys, I am writing the scripts for the EDA and I find this step might be a bit overloading. This scripts require to read in 8 strings, which I think would be super inconvenient when we are actually use it in the terminal commands. I am thinking to breakdown into two parts:
Basic Data Wrangling: where we a) read in the downloaded datasets(2 of them),b) concatenate them together, c) save the concatenated output, d) train test split them e) save train and test df locally
- This requires five strings as inputs when we use terminal command
EDA: where all the graphing happens
- This requtires three strings as inputs when we use terminal command
If no one has other opinion, I'll just continue on this
I see that in the most recent pull request we are changing the helper function in addition to the main analyses code. Just trying to make sure everything runs because the our last submission for milestone some of the test functions didn't run because of the changes in helper function.
I'm bug fixing the script now, some minor issues so far.
Afterwards I'll push it.
We can work in parallel on the next steps when the script/table is added.
Later , I intend to polish it, add any missing functionality, revise the tests and improve the documentation.
Hi Team
I believe we are in a very good shape right now. Our report in HTML is up and running via JupyterBook, our analysis is now fully reproducible on Docker with simple instructions on README.md file, and we have obtained valuable feedbacks from peers.
Despite facing challenges last week, our resilience shone through, and I'm confident we've regained our momentum. This week, let's focus on refining our report by incorporating peer feedback. It's crucial that we meticulously record all modifications in our CHANGELOG.md
file for transparency and tracking.
Keep in mind that each member must at least make at least one change that improves the project in a significant way, and we will be graded on the scale and quality of our changes. I encourage everyone to think outside the box and communicate openly, especially when changes impact multiple team members.
Regarding the Makefile, which accounts for 50% of our Milestone marks, Yingzi is currently leading this effort. Whoever has extra time to spend on our project is encouraged to actively collaborate with her on this task to ensure we all excel in this area.
In closing, I want to express how truly impressed I am with our teamwork and progress. Each challenge we've faced has been a stepping stone, helping us grow stronger and more cohesive as a group. Let's carry this positive momentum forward, learning and succeeding together. I'm excited about our journey ahead and confident in our collective ability to achieve great things.
Hey all!
Relatively minor thing, but wanted to remind everyone to name their branches after the functionality type of their update and some sort of version code or time stamp to differentiate new branches from old branches.
It was mentioned in class and we confirmed later it will effect our grades.
We needed this package to save the PNG files locally, therefore this needs to be added.
If anyone got time, please finish this file and store this file under the report folder for Jupyter book building
Hi team
I have made the following changes to our project to address issues raised by peer review, which have all been documented in changelog.md
. I have fully tested this branch and they work on my PC.
When I ran "docker build --tag group22 ." to create a local docker image with our Dockerfile, I got an error message:
ERROR: failed to solve: process "/bin/bash -o pipefail -c conda install -y python=3.10" did not complete successfully: exit code: 1
Any idea to solve it?
Ok, all the scripts/tables/figures are uploaded. My branch is pending merge.
Next major steps:
I'm reasonably confident I can do a good job of the second task, so unless there's objections I'll work on that now.
I tried to combine wineread.csv and winewhite.csv to be a single csv file. The code for it is: `# Read the CSV files
df1 = pd.read_csv(file1, delimiter=';')
df2 = pd.read_csv(file2, delimiter=';', header=None)
# Concatenate the dataframes
combined_df = pd.concat([df1, df2])`
However, the combined file looks weird. Can anyone tell me how to fix it?
I created a rough draft of the changelog and pushed it to it's own branch.
I broke down what needs to be done as well as assessed who's responsibility it is to fix based on our in person discussion.
Each time you fix something, please edit this changelog with a quick explanation of what you did to fix the issue and where (file location) the fix took place.
We will move the analysis ipynb file into a new folder called notebook
. The src
folder will now house all the custom function scripts and we will make another new folder tests
to house all the pytest scripts.
I have seen this kind of organization from a number of other repos and it seems to be a standard way for good practices. Please let me know if this is a good idea.
As mentioned in Milestone 1 instruction, we need to should use proper grammar and full sentences in your README.MD Point form may occur, but should be less than 10% of your written documents.
This is the summary of our data project:
Our analysis aimed to develop a predictive model to distinguish between red and white wines based on various physicochemical properties. This study employed logistic regression, a model renowned for its balance between predictive power and interpretability.
The regression result suggested that residual sugar and total sulfur dioxide had high positive coefficients, indicating a strong association with white wine, whereas density showed the most substantial negative impact, followed by alcohol and volatile acidity, suggesting these are key indicators of red wine.
The logistic regression model not only achieved high accuracy but also provided valuable insights into the features most indicative of wine type. This model can assist vintners in quality control and classification tasks. Moreover, the interpretability of the model offers a foundation for further research into wine composition and its impact on sensory attributes. Future studies might explore more complex models or delve deeper into feature engineering to enhance predictive accuracy and understanding.
Hey all, I'm writing my assigned tests/functions now.
We decided that I would create functions for the EDA part of our project.
I wanted to double check that there was no overlap with anyone else's work.
The visualizations of the histograms, bar charts ands correlation plot for the EDA is one area I'm concerned might have overlapped.
I only loosely recall what yingzi was working on and that might be part of it?
Can anyone confirm? I'm working on the first part of EDA that's unambiguous while I wait.
I'll go ahead and do the work anyway if I don't hear back by early tomorrow.
Feel free to drop in to discuss the weekly project
I heard that many of our classmates were encountering issues when creating Docker files with the package vegafusion. We might run into this obstacle too. Anyone knows what is the current fix?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.