about • rules • links • project pipeline • tools • data quality • py venv • team

Hackacity is a one-of-a-kind hackathon that explores the potential of city data to develop solutions that will impact the community.

This repository is a mix of ideas, tools and information used to complete the Hackacity 2023 event, in Porto. It contains code developed and graded by a jury on 24/11/2023.

There were already 5 previous editions in Porto, the last one was in 2022. In this edition, there were 34 Teams, 151 Participants, 13 Mentors and 6 jury members in total that participated and worked together at Museu do Carro Elétrico in Porto, Portugal.

- The event starts at 08h30, all the teams should arrive at the venue before 08h15.
- All the participants should remain at the venue during the event.
- All the participants must bring their own laptop or other necessary equipment.
- Your team must deliver the presentation and the produced code by 18:00.
- Presentations should be in .pptx or PDF format.
- Produced code/jupyter notebooks should be sent in a .ZIP folder.

Necessary links for the Hackacity event progress.

• Participants Guidebook OneDrive PDF
• UrbanX Submission Folder OneDrive
• Porto Data Website Data CSV

Information and resources that may help with the project.

• Approaching a data science problem Medium
• Import JSON Data Into Google Spreadsheets Fast Youtube
• How To Create A Correlation Matrix In Excel (With Colors!) Youtube
• Diagrama de Dispersão ou de Correlação Website
• ChatGPT - Python Data Viz Guide ChatGPT

Past hackacity projects.

• hackacity2019 GitHub msramalho
• hackacity-smo-2022 GitHub dfhssilva
• hackacity GitHub tospe

• Correlate electric scooter stop points with scooter accidents.
• Utilize electric scooter GPS to differentiate between road and sidewalk usage.
• Implement traffic lights equipped with cameras, possibly infrared, to optimize red and green signal changes.
• Identify the number of stops within a 1km radius and analyze their (transport) coverage within an x radius.
• Enable direct transportation access to consolidate multiple transports for a single destination.

• Interact with API or download the database. If it's the first, we can use Python or Postman.
• Know what data we are working with. If necessary go to external sources.
• Sanitize data and remove noise data. Depends a lot on the problem.

We should have these programs/tools installed and/or know about them (know they exist):

• Python
• Linux/Windows
• Office 365 (Excel)
• Visual Studio Code
• Jupyter Notebook
• Anaconda
• Pandas
• PEP 8 Style Guide
• Postman
• Google Colab Pro
• RAWGraphs 2.0
• YOLO

• Is data missing? Is it blank? Is that blank because it's optional?

C1 (Pedro) C2 (Tiago) C3 (NULL) C4 ()

• Is the data conformed? What data is stored in a non-standard format?

C1 (27-Out-96) C2 (19961027)

• Is the data unique? Are there repeated/duplicated values? Is that necessary?

C1 (01; Pedro; Oliviera; Portugal) C2 (02; Pedro; Oliviera; Portugal)

• Is the data correct? Are certain values out of date?

C1 (Pedro; 4000-100; Portugal) C2 (Tiago; 4350; Portugal)

• Is the data valid? Spelling mistakes on standard names, roles, bad email well formulated?

C1 (software engineer) C2 (soft engine)

• Is the data consistent? Is there conflicting information?

C1 (01; sonae; pay; 100$; 20:42) C2 (02; sonae; pay; 100$; 20:42)

• Is the primary key (eg.: ID) valid? Is it missing or has an invalid format?

C1 (01; sonae; pay; 100$; 20:42) C2 (NULL; sonae; pay; 100$; 20:42)

We can calculate the percentage of valid data and do a correlation.

1 - (Total Rows / Invalid Rows)

• Waste of time and money;
• Incorrect information and decisions are misguided;
• Future opportunities missed;
• Negative image for the company;
• Lower customer satisfaction;

Why create a virtual environment? We can easily share our requirement packages, so everyone can install and replicate the same environment for the program to work.

Create a virtual environment.

python3 -m venv .venv

Activate a virtual environment.

source .venv/bin/activate

Deactivate a virtual environment.

deactivate

Export all installed packages

python3 -m pip freeze

Using a requirements.txt file to install packages

python3 -m pip install -r requirements.txt