Giter VIP home page Giter VIP logo

empirical_calibration's Introduction

A Python Library For Empirical Calibration

Dealing with biased data samples is a common task across many statistical fields. In survey sampling, bias often occurs due to the unrepresentative samples. In causal studies with observational data, the treated vs untreated group assignment is often correlated with covariates, i.e., not random. Empirical calibration is a generic weighting method that presents a unified view on correcting or reducing the data biases for the tasks mentioned above. We provide a Python library EC to compute the empirical calibration weights. The problem is formulated as a convex optimization and solved efficiently in the dual form. Compared to existing software, EC is both more efficient and robust. EC also accommodates different optimization objectives, supports weight clipping, and allows inexact calibration which improves the usability. We demonstrate its usage across various experiments with both simulated and real-world data.

Paper

Wang, Xiaojing, Miao, Jingang, and Sun, Yunting. (2019). A Python Library For Empirical Calibration. arXiv preprint arXiv:1906.11920.

Installation

The easiest way is propably using pip:

pip install -q git+https://github.com/google/empirical_calibration

If you are using a machine without admin rights, you can do:

pip install -q git+https://github.com/google/empirical_calibration --user

If you are using Google Colab, just add "!" to the beginning:

!pip install -q git+https://github.com/google/empirical_calibration

Package works for python 3.6 or later.

Usage

Package can be imported as

import empirical_calibration as ec

The best way to learn how to use the package is probably by following one of the notebooks, and the recommended way of opening them is Google Colab.

People

Package is created and maintained by Xiaojing Wang, Jingang Miao, and Yunting Sun. Special thanks to Emil Martayan for helping add baseline_weights support.

empirical_calibration's People

Contributors

brianclynch avatar miaojingang avatar wangxj03 avatar yuntingsun avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

empirical_calibration's Issues

Bag: don't need to depend on "enum"

Probably, "enum" isn't needed for this library.

I tried to add this library to below environment.

empirical_calibration git/master  
❯ which python 
/Users/yoshitaka-i/.pyenv/shims/python
                                                                                  
empirical_calibration git/master  
❯ pyenv global       
3.7.3

empirical_calibration git/master  
❯ python --version
Python 3.7.3

When I added this library, I got these error messages.

error log
Installed /Users/yoshitaka-i/.local/lib/python3.7/site-packages/empirical_calibration-0.1-py3.7.egg
Processing dependencies for empirical-calibration==0.1
Searching for enum
Reading https://pypi.org/simple/enum/
Downloading https://files.pythonhosted.org/packages/02/a0/32e1d5a21b703f600183e205aafc6773577e16429af5ad3c3f9b956b07ca/enum-0.4.7.tar.gz#sha256=8c7cf3587eda51008bcc1eed99ea2c331ccd265c231dbaa95ec5258d3dc03100
Best match: enum 0.4.7
Processing enum-0.4.7.tar.gz
Writing /var/folders/bl/64v2k_090rd0z7x3bthx011r0000gp/T/easy_install-c37ahbsr/enum-0.4.7/setup.cfg
Running enum-0.4.7/setup.py -q bdist_egg --dist-dir /var/folders/bl/64v2k_090rd0z7x3bthx011r0000gp/T/easy_install-c37ahbsr/enum-0.4.7/egg-dist-tmp-svotnntw
Traceback (most recent call last):
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
    yield saved
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
    _execfile(setup_script, ns)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
    exec(code, globals, locals)
  File "/var/folders/bl/64v2k_090rd0z7x3bthx011r0000gp/T/easy_install-c37ahbsr/enum-0.4.7/setup.py", line 24, in <module>
    author='Google LLC',
AttributeError: module 'enum' has no attribute '__version__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 37, in <module>
    'typing',
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/__init__.py", line 140, in setup
    return distutils.core.setup(**attrs)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/command/install.py", line 117, in do_egg_install
    cmd.run()
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 415, in run
    self.easy_install(spec, not self.no_deps)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 657, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 704, in install_item
    self.process_distribution(spec, dist, deps)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 749, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/pkg_resources/__init__.py", line 777, in resolve
    replace_conflicting=replace_conflicting
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1060, in best_match
    return self.obtain(req, installer)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1072, in obtain
    return installer(requirement)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 676, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 702, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 887, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 1155, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 1141, in run_setup
    run_setup(setup_script, args)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/sandbox.py", line 253, in run_setup
    raise
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/contextlib.py", line 130, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/contextlib.py", line 130, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/sandbox.py", line 166, in save_modules
    saved_exc.resume()
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/sandbox.py", line 141, in resume
    six.reraise(type, exc, self._tb)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/_vendor/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
    yield saved
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
    _execfile(setup_script, ns)
  File "/Users/yoshitaka-i/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
    exec(code, globals, locals)
  File "/var/folders/bl/64v2k_090rd0z7x3bthx011r0000gp/T/easy_install-c37ahbsr/enum-0.4.7/setup.py", line 24, in <module>
    author='Google LLC',
AttributeError: module 'enum' has no attribute '__version__'

After just removing "enum" from setup.py, I was able to install this library, using this setting. Bug fix: Remove enum from dependency list at setup.py.

Could you add this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.