Giter VIP home page Giter VIP logo

codecarbon's Introduction

banner

Estimate and track carbon emissions from your computer, quantify and analyze their impact.

Documentation


DOI Downloads

About CodeCarbon πŸ’‘

CodeCarbon started with a quite simple question:

What is the carbon emission impact of my computer program? 🀷

We found some global data like "computing currently represents roughly 0.5% of the world’s energy consumption" but nothing on our individual/organisation level impact.

At CodeCarbon, we believe, along with Niels Bohr, that "Nothing exists until it is measured". So we found a way to estimate how much CO2 we produce while running our code.

How?

We created a Python package that estimates your hardware electricity power consumption (GPU + CPU + RAM) and we apply to it the carbon intensity of the region where the computing is done.

calculation Summary

We explain more about this calculation in the Methodology section of the documentation.

Our hope is that this package will be used widely for estimating the carbon footprint of computing, and for establishing best practices with regards to the disclosure and reduction of this footprint.

So ready to "change the world one run at a time"? Let's start with a very quick set up.

Quickstart πŸš€

Installation πŸ”§

From PyPI repository

pip install codecarbon

From Conda repository

conda install -c conda-forge codecarbon

To see more installation options please refer to the documentation: Installation

Start to estimate your impact πŸ“

To get an experiment_id enter:

! codecarbon init

You can now store it in a .codecarbon.config at the root of your project

[codecarbon]
log_level = DEBUG
save_to_api = True
experiment_id = 2bcbcbb8-850d-4692-af0d-76f6f36d79b2 #the experiment_id you get with init

Now you have 2 main options:

Monitoring your machine πŸ’»

In your command prompt use: codecarbon monitor The package will track your emissions independently from your code.

In your Python code 🐍

from codecarbon import track_emissions
@track_emissions()
def your_function_to_track():
  # your code

The package will track the emissions generated by the execution of your function.

There is other ways to use codecarbon package, please refer to the documentation to learn more about it: Usage

Visualize πŸ“Š

You can now visualize your experiment emissions on the dashboard. dashboard

Note that for now, all emissions data send to codecarbon API are public.

Hope you enjoy your first steps monitoring your carbon computing impact! Thanks to the incredible codecarbon community πŸ’ͺ🏼 a lot more options are available using codecarbon including:

  • offline mode
  • cloud mode
  • comet integration...

Please explore the Documentation to learn about it If ever what your are looking for is not yet implemented, let us know through the issues and even better become one of our πŸ¦ΈπŸΌβ€β™€οΈπŸ¦ΈπŸΌβ€β™‚οΈ contributors! more info πŸ‘‡πŸΌ

Contributing 🀝

We are hoping that the open-source community will help us edit the code and make it better!

You are welcome to open issues, even suggest solutions and better still contribute the fix/improvement! We can guide you if you're not sure where to start but want to help us out πŸ₯‡

In order to contribute a change to our code base, please submit a pull request (PR) via GitHub and someone from our team will go over it and accept it.

Check out our contribution guidelines ↗️

Contact @vict0rsch to be added to our slack workspace if you want to contribute regularly!

How To Cite πŸ“

If you find CodeCarbon useful for your research, you can find a citation under a variety of formats on Zenodo.

Here is a sample for BibTeX:

@software{benoit_courty_2024_11171501,
  author       = {Benoit Courty and
                  Victor Schmidt and
                  Sasha Luccioni and
                  Goyal-Kamal and
                  MarionCoutarel and
                  Boris Feld and
                  JΓ©rΓ©my Lecourt and
                  LiamConnell and
                  Amine Saboni and
                  Inimaz and
                  supatomic and
                  Mathilde LΓ©val and
                  Luis Blanche and
                  Alexis Cruveiller and
                  ouminasara and
                  Franklin Zhao and
                  Aditya Joshi and
                  Alexis Bogroff and
                  Hugues de Lavoreille and
                  Niko Laskaris and
                  Edoardo Abati and
                  Douglas Blank and
                  Ziyao Wang and
                  Armin Catovic and
                  Marc Alencon and
                  MichaΕ‚Β StΔ™chΕ‚y and
                  Christian Bauer and
                  Lucas-Otavio and
                  JPW and
                  MinervaBooks},
  title        = {mlco2/codecarbon: v2.4.1},
  month        = may,
  year         = 2024,
  publisher    = {Zenodo},
  version      = {v2.4.1},
  doi          = {10.5281/zenodo.11171501},
  url          = {https://doi.org/10.5281/zenodo.11171501}
}

Contact πŸ“

Maintainers are @vict0rsch @benoit-cty and @SaboniAmine. Codecarbon is developed by volunteers from Mila and the DataForGoodFR community alongside donated professional time of engineers at Comet.ml and BCG GAMMA.

codecarbon's People

Contributors

acatovic avatar acruve15 avatar adijo avatar alencon avatar alexisbogroff avatar benoit-cty avatar dependabot[bot] avatar dsblank avatar edabati avatar goyal-kamal avatar h-sdl avatar inimaz avatar jl-datascientist avatar kngoyal avatar liamconnell avatar lothiraldan avatar lucas-otavio avatar luisblanche avatar marioncoutarel avatar mathilde-leval avatar mstechly avatar mygoditsfull0fstars avatar nikolaskaris avatar ouminasara avatar prmths128 avatar qinganzhao avatar sabamine avatar saboniamine avatar vict0rsch avatar wzziyao avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

codecarbon's Issues

IntelPowerGadget path is incorrect

  • CodeCarbon version: 1.0.0
  • Python version: Any
  • Operating System: Windows

Description

Per @DavidGRX:

In core/cpu.py there is a path (for Windows) to an exe path that was not correct in my case. I understand Python but not the package installation and location, so I replicate my Intel Power directory with the expected name ""C:\Program Files\Intel\Power Gadget 3.5\PowerLog3.0.exe""

We should either clarify the documentation, or add another expected executable path to the python code.

Info logs are messy and confusing

My prints look like this

[2020-11-08 12:21:44,924][apscheduler.executors.default][INFO] - Running job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-11-08 12:21:59 EST)" (scheduled at 2020-11-08 12:21:44.921431-05:00)
[2020-11-08 12:21:44,927][apscheduler.executors.default][INFO] - Job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-11-08 12:21:59 EST)" executed successfully
[2020-11-08 12:21:59,922][apscheduler.executors.default][INFO] - Running job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-11-08 12:22:14 EST)" (scheduled at 2020-11-08 12:21:59.921431-05:00)
[2020-11-08 12:21:59,928][apscheduler.executors.default][INFO] - Job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-11-08 12:22:14 EST)" executed successfully
[2020-11-08 12:22:14,922][apscheduler.executors.default][INFO] - Running job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-11-08 12:22:29 EST)" (scheduled at 2020-11-08 12:22:14.921431-05:00)
[2020-11-08 12:22:14,926][apscheduler.executors.default][INFO] - Job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-11-08 12:22:29 EST)" executed successfully
Done creating display images
Setup done.
Epoch 0:   0%|          | 0/282 [00:00<?, ?batch/s][2020-11-08 12:22:29,922][apscheduler.executors.default][INFO] - Running job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-11-08 12:22:44 EST)" (scheduled at 2020-11-08 12:22:29.921431-05:00)
[2020-11-08 12:22:29,927][apscheduler.executors.default][INFO] - Job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-11-08 12:22:44 EST)" executed successfully
COMET WARNING: Unable to convert model to a string
Epoch 0:   1%|          | 3/282 [00:21<38:43,  8.33s/batch][2020-11-08 12:22:44,922][apscheduler.executors.default][INFO] - Running job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-11-08 12:22:59 EST)" (scheduled at 2020-11-08 12:22:44.921431-05:00)
[2020-11-08 12:22:44,925][apscheduler.executors.default][INFO] - Job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-11-08 12:22:59 EST)" executed successfully
Epoch 0:   2%|▏         | 5/282 [00:30<29:38,  6.42s/batch]%

So we should disable logging by default (this is using comet)

apscheduler info logging

May want to explicitly disable apscheduler info logging as this is pretty annoying every 15 seconds:

[2020-09-23 09:09:34,096][apscheduler.executors.default][INFO] - Running job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-09-23 09:09:34 PDT)" (scheduled at 2020-09-23 09:09:34.095478-07:00)
[2020-09-23 09:09:34,100][apscheduler.executors.default][INFO] - Job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-09-23 09:09:34 PDT)" executed successfully
[2020-09-23 09:09:49,104][apscheduler.executors.default][INFO] - Running job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-09-23 09:09:49 PDT)" (scheduled at 2020-09-23 09:09:49.095478-07:00)
[2020-09-23 09:09:49,106][apscheduler.executors.default][INFO] - Job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-09-23 09:09:49 PDT)" executed successfully

Wonder why that isn't logged at DEBUG level?

Writing emissions.csv to non-default output_dir, emissions are always 0.0

  • CodeCarbon version: 1.0.0
  • Python version: 3.7.9
  • Operating System: MacOS

Description

I've been trying to write emissions.csv to a chosen output_dir (rather than the default working directory) to track emissions data across experiments in the same place. When I do so, I always get emissions of 0.0, but when I write to the default directory this is not the case and I get a small non-zero result. I've found this to be happen across many different experiments.

In the code below, e_path is just the path to the directory I am trying to write to. This may be a mistake on my end, in which case any help is much appreciated, otherwise it looks like a bug.

What I Did

from codecarbon import EmissionsTracker

mnist = tf.keras.datasets.mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0


model = tf.keras.models.Sequential(
    [
        tf.keras.layers.Flatten(input_shape=(28, 28)),
        tf.keras.layers.Dense(128, activation="relu"),
        tf.keras.layers.Dropout(0.2),
        tf.keras.layers.Dense(10),
    ]
)

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])

tracker = EmissionsTracker(project_name='test', output_dir=e_path)
tracker.start()
model.fit(x_train, y_train, epochs=50)
tracker.stop()

emissions.csv is corrupted after second run

If you delete emissions.csv and run the examples/mnist.py then emissions.csv is good.

However, after running multiple times, the appended data is malformed. Here are lines 98 and 99 of the existing file, showing the corrupted data afterwards:

98,2020-07-22T21:26:11,dd0af408-06ea-440a-b99e-6358a9bc5711,project_alpha,48,4,6,Canada,CAN,Ontario,N,,
99,2020-07-24T03:26:11,f65e46fc-298d-4f7a-9827-7f756b1d2f44,project_theta,32,8,5,Brazil,BRA,Sao Paulo,Y,azure,brazilsouth0df9711e-6191-4944-94a4-d9d7e79bf126,2020-11-09T11:08:54,codecarbon,39.154871225357056,2.4245270844359492e-05,0.00011811454117157451,United States,USA,california,N,,
f7518c6e-7c00-43a4-b716-5bb10bd2e923,2020-11-09T11:10:02,codecarbon,38.402961015701294,2.628237527246712e-05,0.00012803860662702362,United States,USA,california,N,,

I think maybe the original file just needs a newline. I'll try that.

Energy Mix Data Refresh

Refresh Energy Mix data to a recent version.

Electricity Map is a possible option to provide near real-time view into the energy mix at even city level from current country-level data.

Enable cloud configuration in Offline Mode

It seems that CloudMetadata.from_utils() wouldnt work if the box isnt connected to the internet. We should enable them to set the cloud provider and region, and in that case leave the country iso code blank.

Usage would be:

tracker = OfflineEmissionsTracker(cloud_provider="AWS", cloud_region="US-east1a")

Clarify Carbonboard emissions units

Carbonboard maps currently display carbon equivalent (kg) for each region/country. This would be better described as the emissions rate (kg/MWh).

Config file(s)

We could have config files to load default params.

Hierarchy could be: args > ./.codecarbon.conf > ~/.codecarbon.conf so that users can store recurring and shared arguments like save_to_file, project_name, output_dir etc.

Thoughts?

Boavizta - Shift collaboration

Hello CodeCarbon team

I was reading your 'call for action' page and I think we could have a discussion together with a working group in France I'm contributing to (https://www.boavizta.org - in French sorry). I'm also a contributor to the Shift project for their 'Digital sobriety report ' (https://theshiftproject.org/article/deployer-la-sobriete-numerique-rapport-shift/ the last one soon to be translated).

One common topic we have for instance is the world electricity mix I believe you are trying to evaluate on the full scope (inc upstream-downstream effects) : is someone of your team would be ok for a chat in the coming weeks ?

Thanks
Arnaud Gueguen

Dash app crashes when you clear selection

Dash app crashes when you clear the select list:

Screenshot from 2020-11-09 11-05-52

[2020-11-09 11:05:43,804] ERROR in app: Exception on /_dash-update-component [POST]
Traceback (most recent call last):
  File "/home/dblank/.local/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/dblank/.local/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/dblank/.local/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/dblank/.local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/dblank/.local/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/dblank/.local/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/dblank/.local/lib/python3.8/site-packages/dash/dash.py", line 1059, in dispatch
    response.set_data(func(*args, outputs_list=outputs_list))
  File "/home/dblank/.local/lib/python3.8/site-packages/dash/dash.py", line 994, in add_context
    output_value = func(*args, **kwargs)  # %% callback invoked %%
  File "../codecarbon/viz/carbonboard.py", line 77, in update_project_data
    project_summary = data.get_project_summary(project_data.data)
  File "/home/dblank/comet/code-carbon/codecarbon/viz/data.py", line 24, in get_project_summary
    last_run = project_data[-1]
IndexError: list index out of range

emissions is 0.0 vs null

When I run an experiment with codecarbon without a GPU, I get the resulting emissions.csv data:

experiment_id,timestamp,project_name,duration,emissions,energy_consumed,country_name,country_iso_code,region,on_cloud,cloud_provider,cloud_region
51d77d04-33a7-4ca6-b55a-b157b0262e7b,2020-09-18T13:00:51,carbon-footprint,51.45621919631958,0.0,0.0,United States,USA,california,N,,

I thought the emissions value might be one that I could tell that it didn't record (for example null) and then I could fake a result based on the time the experiment ran and typical CPU usage in the USA or world. But I can't tell the difference between having a GPU with zero emissions, vs not having a GPU at all.

Any ideas what we can do here?

Add Canada regional map to carbonboard

Now that we have Canadian Energy Mix data, we can display its regional map like the USA. I suggest that we use a dropdown to select which country the user would like to see a regional breakdown for

Code fails silently when no hardware is detected

  • CodeCarbon version: 1.0.0
  • Python version: 3.7.5
  • Operating System: WSL (used under MS Windows 10)

Description

It seems when used under WSL, Intel Power Gadget and RAPL are not available. So you run the start()/stop() code and everything seems fine, but the emissions file is never written. There are probably lot of WSL users running jupyter notebooks and other DS/ML code under Windows in this way. It would be good to actually check that hardware is present.

It can easily be fixed in BaseEmissionsTracker class initialization code with the following:

if len(self._hardware) == 0:
sys.exit("No hardware detected")

You need to import sys at the beginning, and you should place the above code after all your if/else checks on GPUs/CPUs.

What I Did

from codecarbon import OfflineEmissionsTracker
tracker = OfflineEmissionsTracker(country_iso_code="CAN")
tracker.start()

GPU intensive training code

tracker.stop()

Change data format to JSON

This would allow the logging of individual hardware measurements:

[
	{
		"id": exp_id,
		"duration": ...,
		"emissions":{
						"hardware1": {
										"emissions": ...,
										"energy": ...,
									 },
						"hardware2": {...}
					},
		"region": region,
		...
	},
	{...},
	{...}
]

Thoughts @kngoyal @Lothiraldan ?

Background scheduling requires the GIL

By working on the integration between code-carbon and the Comet Python SDK, I saw that the apscheduler BackgroundScheduler was using threads under the hood. This might causes issues with some machine learning libraries that execute some C code during a long time without releasing the GIL, we had a similar issue in the past.

Here is a reproduction script which runs in about 3 minutes and 40 seconds on my laptop with Python 3.7.5. I am using re.match for mimicking a piece of code holding the GIL for a certain amount of time:

import time
import re
from codecarbon import EmissionsTracker

tracker = EmissionsTracker()
tracker.start()

time.sleep(15)

re.match(r"(a?){32}a{32}", "a" * 32)

time.sleep(15)

tracker.stop()

When running with the following diff:

diff --git a/codecarbon/emissions_tracker.py b/codecarbon/emissions_tracker.py
index 7b07ea2..66283d7 100644
--- a/codecarbon/emissions_tracker.py
+++ b/codecarbon/emissions_tracker.py
@@ -155,6 +155,7 @@ class BaseEmissionsTracker(ABC):
         every `self._measure_power` seconds.
         :return: None
         """
+        print("Measure power", time.time())
         self._total_energy += Energy.from_power_and_time(
             power=self._hardware.total_power,
             time=Time.from_seconds(self._measure_power_secs),

Here is the output that highlight the fact that the background job was not scheduled while the re.match was running:

Measure power 1599575654.264812
Run time of job "BaseEmissionsTracker._measure_power (trigger: interval[0:00:15], next run at: 2020-09-08 16:34:29 CEST)" was missed by 0:00:04.694579
Measure power 1599575834.2660286

I think we need to clarify which level of precision that we need.

emissions always equal to 0

  • CodeCarbon version:1.0.0
  • Python version:3.7.4
  • Operating System:windows 10

Description

I have run the following code on my laptop
###########################
import tensorflow as tf

from codecarbon import EmissionsTracker

mnist = tf.keras.datasets.mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential(
[
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(128, activation="relu"),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10),
]
)

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])

tracker = EmissionsTracker()
tracker.start()
model.fit(x_train, y_train, epochs=1000)
emissions: float = tracker.stop()
print(emissions)
#################################

But the emissssion is always equal to 0.0

What does this result mean?

Carbon tracking for shared resources

Say I run a program on a node with two GPUs, each with 32 CPUs. The node is shared. If my program runs on 1 GPU only, 32 CPUs, will codecarbon track the emissions of these resources only or of the whole node ?

Invalid characters in RAPL test files for Windows

  • CodeCarbon version: master (05ae023)
  • Python version: 3.8.6
  • Operating System: Windows 10

Description

RAPL test files contains invalid characters for Windows (: colons), which trigger an error when trying to checkout the repository on windows:

$ git pull
error: invalid path `tests/test_data/rapl/intel-rapl:0/contraint_0_max_power_uw`
error: invalid path `tests/test_data/rapl/intel-rapl:0/contraint_0_name`
...

Could we remove those files and create them dynamically during the tests?

Leave logging configuration to client programs

  • CodeCarbon version: 1.0.0
  • Python version: 3.8.6
  • Operating System: Linux (with Anaconda)

Description

I'm using CodeCarbon in a program that configures the Python logging infrastructure to provide output in my desired format.

CodeCarbon also configures Python logging, with basicConfig, on import. This results in duplicate logging configurations, so my program's logging setup is no longer respected and messages are duplicated (since both my handler and the basicConfig handler are printing them).

As a user of Python logging, I prefer that libraries allow me to configure logging myself, providing a hook if needed, rather than overriding logging configuration at import.

Write periodically

@kngoyal @Lothiraldan right now if the code fails the tracker does not write to emissions.csv, I cannot see the emissions for my comet exp which failed un-gracefully.

What do you think is best?

  1. write periodically so that no matter what we have data
  2. use atexit.register

@Lothiraldan how does comet handles the end/crash of a python script? So that codecarbon can write to the file before the last comet upload?

Same as ContextualVersionConflict exception in Google Colab issue #108

  • CodeCarbon version:1.0.0
  • Python version: 3.6.9
  • Operating System: Ubuntu 18.04.5 LTS (Bionic Beaver)

Description

Describe what you were trying to get done. Your example
Tell us what happened, what went wrong, and what you expected to happen.

Example works fine before 2021/01/15 never after

What I Did

Your example

!pip install codecarbon

import tensorflow as tf

from codecarbon import EmissionsTracker

mnist = tf.keras.datasets.mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0


model = tf.keras.models.Sequential(
    [
        tf.keras.layers.Flatten(input_shape=(28, 28)),
        tf.keras.layers.Dense(128, activation="relu"),
        tf.keras.layers.Dropout(0.2),
        tf.keras.layers.Dense(10),
    ]
)

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])

tracker = EmissionsTracker()
tracker.start()
model.fit(x_train, y_train, epochs=10)
emissions: float = tracker.stop()
print(emissions)_

Paste the command(s) you ran and the output.
If there was a crash, please include the traceback here.

Collecting codecarbon
  Downloading https://files.pythonhosted.org/packages/1d/78/0f6f06250558ca5dc21abb28660fecb78f09b4d5258d8e634393a780d1a6/codecarbon-1.0.0-py3-none-any.whl (49kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 51kB 2.8MB/s 
Collecting APScheduler
  Downloading https://files.pythonhosted.org/packages/5d/e9/b5e4f7aaea076bbfb241b04824c63cd42fe8029358103c27375e690bddc0/APScheduler-3.7.0-py2.py3-none-any.whl (59kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 61kB 4.1MB/s 
Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from codecarbon) (2.23.0)
Collecting dash
  Downloading https://files.pythonhosted.org/packages/bc/b4/0bd5c94fdcb0eccb93c3c8068fe10f5607e542337d0b8f6e2d88078316a9/dash-1.19.0.tar.gz (75kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 81kB 3.9MB/s 
Collecting fire
  Downloading https://files.pythonhosted.org/packages/11/07/a119a1aa04d37bc819940d95ed7e135a7dcca1c098123a3764a6dcace9e7/fire-0.4.0.tar.gz (87kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 92kB 4.7MB/s 
Requirement already satisfied: dataclasses in /usr/local/lib/python3.6/dist-packages (from codecarbon) (0.8)
Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (from codecarbon) (1.1.5)
Collecting dash-bootstrap-components
  Downloading https://files.pythonhosted.org/packages/1d/26/17f8b18a79bae9e278b7674ad0aef8007e87b4d2280525a95103fd2a8fe5/dash_bootstrap_components-0.11.1-py2.py3-none-any.whl (187kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 194kB 6.4MB/s 
Collecting pynvml
  Downloading https://files.pythonhosted.org/packages/1b/1a/a25c143e1d2f873d67edf534b269d028dd3c20be69737cca56bf28911d02/pynvml-8.0.4-py3-none-any.whl
Requirement already satisfied: six>=1.4.0 in /usr/local/lib/python3.6/dist-packages (from APScheduler->codecarbon) (1.15.0)
Requirement already satisfied: pytz in /usr/local/lib/python3.6/dist-packages (from APScheduler->codecarbon) (2018.9)
Collecting tzlocal~=2.0
  Downloading https://files.pythonhosted.org/packages/5d/94/d47b0fd5988e6b7059de05720a646a2930920fff247a826f61674d436ba4/tzlocal-2.1-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=0.7 in /usr/local/lib/python3.6/dist-packages (from APScheduler->codecarbon) (53.0.0)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->codecarbon) (1.24.3)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->codecarbon) (2.10)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->codecarbon) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->codecarbon) (2020.12.5)
Requirement already satisfied: Flask>=1.0.4 in /usr/local/lib/python3.6/dist-packages (from dash->codecarbon) (1.1.2)
Collecting flask-compress
  Downloading https://files.pythonhosted.org/packages/b2/7a/9c4641f975fb9daaf945dc39da6a52fd5693ab3bbc2d53780eab3b5106f4/Flask_Compress-1.8.0-py3-none-any.whl
Requirement already satisfied: plotly in /usr/local/lib/python3.6/dist-packages (from dash->codecarbon) (4.4.1)
Collecting dash_renderer==1.9.0
  Downloading https://files.pythonhosted.org/packages/be/a6/dd1edfe7b1102274e93991736c35b2a5e1a63b524c8d9f41bbb30f17340b/dash_renderer-1.9.0.tar.gz (1.0MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1.0MB 7.2MB/s 
Collecting dash-core-components==1.15.0
  Downloading https://files.pythonhosted.org/packages/22/78/ae0829e673f3df77403bcdb35073b1ed2f156080f5bcac6f21c1047d73fe/dash_core_components-1.15.0.tar.gz (3.5MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 3.5MB 13.2MB/s 
Collecting dash-html-components==1.1.2
  Downloading https://files.pythonhosted.org/packages/3f/25/56de2708128fe375eecc2e18e0ccdc3a853494966e36334ec8a30be99b94/dash_html_components-1.1.2.tar.gz (188kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 194kB 40.6MB/s 
Collecting dash-table==4.11.2
  Downloading https://files.pythonhosted.org/packages/3d/ae/96cb64b58d76391604b57f8c747f9a19ab2122e7ba214e2e0cf35484962b/dash_table-4.11.2.tar.gz (1.8MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1.8MB 38.3MB/s 
Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from dash->codecarbon) (0.16.0)
Requirement already satisfied: termcolor in /usr/local/lib/python3.6/dist-packages (from fire->codecarbon) (1.1.0)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.6/dist-packages (from pandas->codecarbon) (2.8.1)
Requirement already satisfied: numpy>=1.15.4 in /usr/local/lib/python3.6/dist-packages (from pandas->codecarbon) (1.19.5)
Requirement already satisfied: itsdangerous>=0.24 in /usr/local/lib/python3.6/dist-packages (from Flask>=1.0.4->dash->codecarbon) (1.1.0)
Requirement already satisfied: click>=5.1 in /usr/local/lib/python3.6/dist-packages (from Flask>=1.0.4->dash->codecarbon) (7.1.2)
Requirement already satisfied: Jinja2>=2.10.1 in /usr/local/lib/python3.6/dist-packages (from Flask>=1.0.4->dash->codecarbon) (2.11.2)
Requirement already satisfied: Werkzeug>=0.15 in /usr/local/lib/python3.6/dist-packages (from Flask>=1.0.4->dash->codecarbon) (1.0.1)
Collecting brotli
  Downloading https://files.pythonhosted.org/packages/b4/d3/7c98f05b7b9103e2f3a112ba42f269c798155b3e5404fb80bb8f823aaebe/Brotli-1.0.9-cp36-cp36m-manylinux1_x86_64.whl (357kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 358kB 41.2MB/s 
Requirement already satisfied: retrying>=1.3.3 in /usr/local/lib/python3.6/dist-packages (from plotly->dash->codecarbon) (1.3.3)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from Jinja2>=2.10.1->Flask>=1.0.4->dash->codecarbon) (1.1.1)
Building wheels for collected packages: dash, fire, dash-renderer, dash-core-components, dash-html-components, dash-table
  Building wheel for dash (setup.py) ... done
  Created wheel for dash: filename=dash-1.19.0-cp36-none-any.whl size=84013 sha256=08dc3f0960a54ea6da81e46a8b2a86f2edbe642fc90919781281d6a8886e4dee
  Stored in directory: /root/.cache/pip/wheels/f7/50/a7/a230ff7f503b10120bff18c2524a375bb85a61ce6b519c8a77
  Building wheel for fire (setup.py) ... done
  Created wheel for fire: filename=fire-0.4.0-py2.py3-none-any.whl size=115928 sha256=98b65b034d06b4f56aa4bb96d6f370f73979c28c85c859735497f9730ea62c04
  Stored in directory: /root/.cache/pip/wheels/af/19/30/1ea0cad502dcb4e66ed5a690279628c827aea38bbbab75d5ed
  Building wheel for dash-renderer (setup.py) ... done
  Created wheel for dash-renderer: filename=dash_renderer-1.9.0-cp36-none-any.whl size=1014869 sha256=50e43efd2f7bce53545b685a442ae465944268e14b39046367d6bf9065392c26
  Stored in directory: /root/.cache/pip/wheels/46/a0/ec/2be2e8fc750e623d76f9690c397cc5ab28b33d0a16a49e10c5
  Building wheel for dash-core-components (setup.py) ... done
  Created wheel for dash-core-components: filename=dash_core_components-1.15.0-cp36-none-any.whl size=3527014 sha256=3aa1d57e14283ec396209981d532ce23ccc4797878ad6bccbbc0382cdfe67191
  Stored in directory: /root/.cache/pip/wheels/53/3d/be/d628d6f66eedf9597f0c89c8ff43a5020ad1c25152c77d8e9f
  Building wheel for dash-html-components (setup.py) ... done
  Created wheel for dash-html-components: filename=dash_html_components-1.1.2-cp36-none-any.whl size=427830 sha256=f8dd0350e4bbd15712dc7b37d39081308888de23f12117990ed289595fc9be13
  Stored in directory: /root/.cache/pip/wheels/44/95/70/0dc41f9b4e31b8a7ea22193aad5647b2c85cfab37bf13c0242
  Building wheel for dash-table (setup.py) ... done
  Created wheel for dash-table: filename=dash_table-4.11.2-cp36-none-any.whl size=1839870 sha256=d164847d466bc4fc01a6162506ce077e1e4ee4f8831b83b176c153e5d351f9fe
  Stored in directory: /root/.cache/pip/wheels/72/d9/f6/2ad62ac0037f1f0c0d3d10948a596b594a069057df0656ac3f
Successfully built dash fire dash-renderer dash-core-components dash-html-components dash-table
Installing collected packages: tzlocal, APScheduler, brotli, flask-compress, dash-renderer, dash-core-components, dash-html-components, dash-table, dash, fire, dash-bootstrap-components, pynvml, codecarbon
  Found existing installation: tzlocal 1.5.1
    Uninstalling tzlocal-1.5.1:
      Successfully uninstalled tzlocal-1.5.1
Successfully installed APScheduler-3.7.0 brotli-1.0.9 codecarbon-1.0.0 dash-1.19.0 dash-bootstrap-components-0.11.1 dash-core-components-1.15.0 dash-html-components-1.1.2 dash-renderer-1.9.0 dash-table-4.11.2 fire-0.4.0 flask-compress-1.8.0 pynvml-8.0.4 tzlocal-2.1
---------------------------------------------------------------------------
ContextualVersionConflict                 Traceback (most recent call last)
<ipython-input-2-43bdafe14602> in <module>()
      3 import tensorflow as tf
      4 
----> 5 from codecarbon import EmissionsTracker
      6 
      7 mnist = tf.keras.datasets.mnist

6 frames
/usr/local/lib/python3.6/dist-packages/codecarbon/__init__.py in <module>()
      3 """
      4 
----> 5 from .emissions_tracker import (
      6     EmissionsTracker,
      7     OfflineEmissionsTracker,

/usr/local/lib/python3.6/dist-packages/codecarbon/emissions_tracker.py in <module>()
     13 from typing import Callable, List, Optional
     14 
---> 15 from apscheduler.schedulers.background import BackgroundScheduler
     16 
     17 from codecarbon.core import cpu, gpu

/usr/local/lib/python3.6/dist-packages/apscheduler/__init__.py in <module>()
      2 
      3 try:
----> 4     release = get_distribution('APScheduler').version.split('-')[0]
      5 except DistributionNotFound:
      6     release = '3.5.0'

/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py in get_distribution(dist)
    464         dist = Requirement.parse(dist)
    465     if isinstance(dist, Requirement):
--> 466         dist = get_provider(dist)
    467     if not isinstance(dist, Distribution):
    468         raise TypeError("Expected string, Requirement, or Distribution", dist)

/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py in get_provider(moduleOrReq)
    340     """Return an IResourceProvider for the named module or requirement"""
    341     if isinstance(moduleOrReq, Requirement):
--> 342         return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
    343     try:
    344         module = sys.modules[moduleOrReq]

/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py in require(self, *requirements)
    884         included, even if they were already activated in this working set.
    885         """
--> 886         needed = self.resolve(parse_requirements(requirements))
    887 
    888         for dist in needed:

/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py in resolve(self, requirements, env, installer, replace_conflicting, extras)
    775                 # Oops, the "best" so far conflicts with a dependency
    776                 dependent_req = required_by[req]
--> 777                 raise VersionConflict(dist, req).with_context(dependent_req)
    778 
    779             # push the new requirements onto the stack

ContextualVersionConflict: (tzlocal 1.5.1 (/usr/local/lib/python3.6/dist-packages), Requirement.parse('tzlocal~=2.0'), {'APScheduler'})

Pre-commit fail

see : https://github.com/mlco2/codecarbon/runs/1424483572

Error: Unable to process command '::set-env name=PY::d874f77ec1348dfde58a9317309117a9df04cdc975815e70e2910a54b56d97ae' successfully.
Error: The `set-env` command is disabled. Please upgrade to using Environment Files or opt into unsecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_COMMANDS` environment variable to `true`. For more information see: github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands

Any idea @Lothiraldan ?

Document `utils`

Functions like codecarbon.utils.gpu.get_gpu_static_info() could be useful. If this comes from comet's gpu utils maybe just importing the package and relying on their doc would ease that. Toughts?

Fail gracefully

Failure in Code Carbon should not fail the actual user code base.

Visualization Tool: alignment and rounding

For the metrics, Power Consumption... through Last Run Carbon.... the alignment is wonky. The first and third lines are left-justified, second and fourth are right-justified. Also, I'm thinking you could probably round these values somewhat?

Screenshot from 2020-10-28 12-59-27

Add Canadian Provinces Energy Mix Data

Currently, the package leverages energy mix data for US at state level and for other countries at the country level.
Add energy mix at the provincial level for Canada.

ContextualVersionConflict exception in Google Colab

  • CodeCarbon version: 1.0.0
  • Python version: 3.6.9
  • Operating System: Linux 2d470bfad1b2 4.19.112+

Description

Hello,
I hope this is the right place for such question as it is my first time raising an issue on github.
I'm using codecarbon on Google Colab to measure the impact of an AI's training.
I started using it around Dec 2020 and did not have problems with it until last week without any change to my notebook.
Now it is impossible to import codecarbon, it raise a ContextualVersionConflict.

What I Did

On Google Colab I try to run the following commands:

!pip install codecarbon
from codecarbon import EmissionsTracker

Which ends up with the following error message:

---------------------------------------------------------------------------
ContextualVersionConflict                 Traceback (most recent call last)
<ipython-input-1-9941a1808b4c> in <module>()
      1 get_ipython().system('pip install codecarbon')
      2 
----> 3 from codecarbon import EmissionsTracker

6 frames
/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py in resolve(self, requirements, env, installer, replace_conflicting, extras)
    775                 # Oops, the "best" so far conflicts with a dependency
    776                 dependent_req = required_by[req]
--> 777                 raise VersionConflict(dist, req).with_context(dependent_req)
    778 
    779             # push the new requirements onto the stack

ContextualVersionConflict: (tzlocal 1.5.1 (/usr/local/lib/python3.6/dist-packages), Requirement.parse('tzlocal~=2.0'), {'APScheduler'})

Here is the complete log of the installation.

After some research over this issue, I tried to manually add the tzlocal installation before codecarbon's with this command

!pip install tzlocal~=2.0

But the result is the same.
I also tried to reset the environment, or with another account and I still have the issue.

At first I thought this was due to a new release of CodeCarbon, but since I see none, do you have any idea how to make usable again on Colab ?

Thanks

Add Computation Functions

Add Linear and Exponential Functions to verify the relationship between Compute Time and Emissions Generated

Separate HTML, CSS as much as possible

In would be very handy if the HTML and CSS were separate, or if they could be generated.

It would be very useful to have the HTML in a form similar to:

<html>
  <body style="">
    <div id="react-entry-point">
      <div class="container" style="padding-top: 50px;">
        <div class="jumbotron">
          <h1 style="text-align: center;">Carbon Footprint</h1>
          <p class="lead" style="text-align: center; padding-left: 0.5%;">Measure Compute Emissions</p>
        </div>
        <div style="padding-left: 1.4%; padding-right: 1.4%;">
          <h2 style="text-align: center;">Across All Projects</h2>
          <h4 style="text-align: left; float: left;">Net Power Consumption : 
            <strong id="net_power_consumption" style="font-weight: normal; color: green;">
            </strong>
          </h4>
          <h4 style="text-align: right; float: right;">Net Carbon Equivalent : 
            <strong id="net_carbon_equivalent" style="font-weight: normal; color: green;">
            </strong>
          </h4>
        </div>
        <div style="display: inline-block;" class="col">
          <br>
          <h3 style="text-align: left;">Select an Experiment</h3>
          <div>
            <div id="chart-mount" class="dash-dropdown">
            </div>
          </div>
        </div>
        <div>
          <div style="padding-left: 1.4%; padding-right: 1.4%;"><br>
            <div>
              <h3 style="float: left;">Infrastructure Hosted at 
                <strong id="project_infrastructure_location" style="font-weight: normal; color: green;">
                </strong>
              </h3>
            </div><br><br>
            <div>
              <h4 style="float: left;">Power Consumption Across All Experiments : 
                <strong id="project_power_consumption" style="font-weight: normal; color: green;">
                </strong>
              </h4>
              <h4 style="float: right;">Last Run Power Consumption : 
                <strong id="last_run_power_consumption" style="font-weight: normal; color: green;">
                </strong>
              </h4>
            </div><br>
            <div><h4 style="float: left;">Carbon Equivalent Across All Experiments : 
              <strong id="project_carbon_equivalent" style="color: green; font-weight: bold;">
              </strong>
              </h4>
              <h4 style="float: right;">Last Run Carbon Equivalent : 
                <strong id="last_run_carbon_equivalent" style="font-weight: normal; color: green;">
                </strong>
              </h4>
            </div>
          </div>
          <div><br><br><br><br>
            <div style="display: inline-block;">
              <h2 style="text-align: center;">Exemplary Equivalents</h2><br>
              <div style="float: left; width: 25%;">
                <img id="house_icon" style="height: 20%; width: 50%;" src="https://raw.githubusercontent.com/mlco2/code-carbon/master/codecarbon/viz/assets/house_icon.png">
                <div><strong id="household_fraction" style="color: green; font-size: 20px; padding-left: 4%;">
                  </strong>
                  <h5 style="padding-left: 3.5%;">of weekly</h5>
                  <h5 style="padding-left: 2.8%;">American</h5>
                  <h5>household</h5>
                  <h5 style="padding-left: 1.4%;">emissions</h5>
                </div>
              </div>
              <div style="float: left; width: 50%; padding-left: 90px;">
                <img id="car_icon" style="height: 43%; width: 28.5%;" src="https://raw.githubusercontent.com/mlco2/code-carbon/master/codecarbon/viz/assets/car_icon.png">
                <div>
                  <strong id="car_miles" style="color: green; font-weight: bold; font-size: 20px;">
                  </strong>
                  <h5 style="padding-left: 5.5%;">driven</h5></div>
              </div>
              <div style="float: right; width: 25%;">
                <img id="tv_icon" style="height: 35%; width: 53%; padding-left: 5%;" src="https://raw.githubusercontent.com/mlco2/code-carbon/master/codecarbon/viz/assets/tv_icon.png">
                <div>
                  <strong id="tv_time" style="color: green; font-size: 20px; padding-left: 8%;">
                  </strong>
                  <h5 style="padding-left: 5%;">of 32-inch</h5>
                  <h5 style="padding-left: 10%;">LCD TV</h5>
                  <h5 style="padding-left: 6.4%;">watched</h5>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
    </div>
  </body>
</html>

and the CSS removed from the HTML and turned into a stylesheet, like:

.lead {
  text-align: center; 
  padding-left: 0.5%;
}
h5 {
  padding-left: 6.4%;
}

In order for the CSS to work, some classes may need to be added to the HTML elements.

Linux RAPL files not present on AWS EC2 instances

  • CodeCarbon version: 1.0.0
  • Python version: Any
  • Operating System: Ubuntu 18.04, AWS Linux
    - They use Linux 4.15 and 4.14 respectively

Description

I have tried to use the tool on two Linux OS's on AWS EC2 instances and it seems that the RAPL file is not found. cpu.is_rapl_available() returns False. I'm assuming that this is due to sys/class/ not having a powercap/ directory as specified in the codebase.

codecarbon as a server

Although I think that the emission tracker and dash website should be two different programs, since they are one, we should make it easy to use.

Currently the docs suggest that you need to have the source code to run the server:

python codecarbon/viz/carbonboard.py --filepath="./examples/emissions.csv"

But that isn't necessary. It is easier to:

python -m codecarbon.viz.carbonboard --filepath="./examples/emissions.csv"

But even that isn't that easy. We could easily make it do:

codecarbon --server --filepath="./examples/emissions.csv"

or at least

python -m codecarbon --server --filepath="./examples/emissions.csv"

Also, that gets rid of yet another name variation (carbonboard).

Sweden ISO code incorrectly specified in global_energy_mix.json

  • CodeCarbon version: 1.0.0
  • Python version: 3.7.5
  • Operating System: WSL (used under MS Windows 10)

Description

Sweden ISO code should be "SWE" and not "SWI", lines 579 and 585 in data/private_infra/2016/global_energy_mix.json

What I Did

from codecarbon import track_emissions
@track_emissions(offline=True, country_iso_code="SWE")

Geojs API could not returns Region

I was testing the Comet integration with codecarbon and saw the following error:

logs/log-784272.log:Traceback (most recent call last):
logs/log-784272.log-  File "/home/project/comet-python-client/comet-client-lib/comet_ml/experiment.py", line 616, in _on_end
logs/log-784272.log-    self._co2_tracker.stop()
logs/log-784272.log-  File "/home/project/code-carbon/codecarbon/emissions_tracker.py", line 97, in stop
logs/log-784272.log-    geo: GeoMetadata = self._get_geo_metadata()
logs/log-784272.log-  File "/home/project/code-carbon/codecarbon/emissions_tracker.py", line 209, in _get_geo_metadata
logs/log-784272.log-    return GeoMetadata.from_geo_js(self._data_source.geo_js_url)
logs/log-784272.log-  File "/home/project/code-carbon/codecarbon/external/geography.py", line 77, in from_geo_js
logs/log-784272.log-    region=response["region"].lower(),
logs/log-784272.log-KeyError: 'region'

It seems that getojs doesn't always returns region, here is the answer payload from https://get.geojs.io/v1/ip/geo.json locally:

{
  "organization_name": "SFR SA",
  "accuracy": 100,
  "asn": 15557,
  "organization": "AS15557 SFR SA",
  "timezone": "Europe/Paris",
  "longitude": "2",
  "country_code3": "FRA",
  "area_code": "0",
  "ip": "REDACTED",
  "country": "France",
  "continent_code": "EU",
  "country_code": "FR",
  "latitude": "46"
}

Do we absolutely need the region?

Issue with rapl energy on Linux - value seems wrong by a 100x factor

  • CodeCarbon version: codecarbon==1.0.0
  • Python version: 3.8.3
  • Operating System: Linux ubuntu 20.04 LTS

Description

I've been measuring the energy consumption of a CPU-only inference (7 min long) using several tools :

Codecarbon gives wildly different results : 2.15 Wh for powerAPI, 2.21 Wh for pyJoules and 0.035 Wh for codecarbon. Given the duration of the test and the processor in my test computer, I believe powerAPI and pyJoule to be right and codecarbon to have an issue.
I had a quick look on how codecarbon measure energy on linux, in IntelRAPL.get_cpu_details ; if I understand it correctly, you calculate the power of the cpu by reading the energy consumed during 10 milliseconds (and of course dividing it then by 10ms, to get an average power during these 10ms) and then use that value to compute the energy for the tracker interval (15s by default). This seems to lead to under-estimating the global energy usage, there is no reason to be sure that the average power these 10ms is the same than the average power during the 10s interval.
Besides, as the rapl counter simply counts joules, it would be easier and more accurate to look at this counter simply at the beginning and and of the 10s interval to get a measure of the energy used by the cpu during that interval.

Is there a reason for this process or have I misunderstood the implementation ?

I have not tried codecarbon on Windows so I can not say if the reported value has the same issue.

Use PyPI Extras to reduce dependencies in server-based use

  • CodeCarbon version: 1.0
  • Python version: 3.8

Description

Another feature request πŸ™‚.

When installing CodeCarbon from PyPI, it pulls in Dash, Flask, Plotly, and several other packages that - as far as I know - are only needed for visualizing results, not for tracking and recording them.

Putting these dependencies behind a PyPI 'extra' would reduce the installation load when only using CodeCarbon to output CSV files for later analysis.

Expose more emissions statistics in API

  • CodeCarbon version: 1.0
  • Python version: 3.8
  • Operating System:

Description

I would love to easily be able to get the kWh, not just the kG, in my code when calling EmissionsTracker. This is likely a backwards-incompatible change, but given that I'm not at all sure CodeCarbon has my building's power mix cost correct, I'm love to be able to log kWh, or to have access to any of CodeCarbon's statistics, when shutting down the tracker. I can get the data out of emissions.csv, but stop() just returns the kG of carbon, and the full energy consumption / emissions statistics aren't stored in a field or anything I can programmatically access.

Just an idea. But it would make things a little easier for how I'm looking at integrating CodeCarbon into my workflows.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.