Linux/MacOS: Windows: Coverage:

• Our CGO'07 paper (which later motivated creation of CK) received CGO test of time award;
• Demo of CK-powered AI API with collaboratively optimized DNN is now available here;
• CK-powered artifact received distinguished award at CGO'17 (Austin,TX): paper with AE appendix and CK workflow, GitHub sources, CK interactive dashboard snapshot;
• Our CK-related CNRS presentation "Enabling open and reproducible research at computer systems conferences: the good, the bad and the ugly"
• Michel Steuwer (University of Edinburgh) blogged about CK concepts;
• We discussed with the commnunity how to use CK to improve Artifact Evaluation for CGO,PPoPP,PACT,RTSS,SC;
• Check out new CK-AI initiative to provide common JSON API and web-services for popular DNN frameworks (Caffe, TensorFlow, TensorRT) and prepare them for collaborative optimization (code and models);
• ARM uses CK as a front-end for systematic and reproducible benchmarking and tuning of real workloads: GitHub, ARM TechCon'16 presentation;
• CK-powered open challenges in computer engineering have been updated: link;
• Fully updated documentation is now available: Wiki, Getting Started Guide, Portable workflows;
• We have moved Open Science resources here;
• We have added continuous integration for Linux and Windows;
• General Motors and dividiti crowdsource benchmarking and optimization of CAFFE (DNN framework) using CK: GitHub, Android app;
• CK supports Jupyter notebooks and Docker;
• cTuning foundation and ARM has received HiPEAC technology transfer award for the CK concept.

Collective Knowledge is our "swiss knife" for open, collaborative and reproducible experimentation. CK is a small, portable and customizable research platform to

• share artifacts as reusable and indexable Python components with unified JSON API and meta information (programs, benchmarks, data sets, tools, predictive models, etc) - see Artifact Evaluation website;
• automate detection or installation of all software dependencies on Linux, Windows, MacOS, Android while enabling co-existence of different versions (compilers, benchmarks, libraries, tools) for a given experimental workflow across diverse Linux, Windows and Android based platforms via CK cross-platform customizable package and environment manager;
• quickly prototype experimental workflows from shared components (such as customizable and multi-objective autotuning for DSL, OpenCL, CUDA, MPI, OpenMP and compiler flags) across diverse hardware and software (CK repo);
• crowdsource experiments across diverse hardware and workloads provided by volunteers and validate ideas or report unexpected behavior and bugs (CK repo);
• reuse and collaborative improve JSON description of all existing platforms from IoT to supercomputers (CK repo);
• abstract access to continuously evolving software and hardware stack (CK repo);
• automate, reproduce and crowdsource empirical experiments using CK JSON-based web services (live repo);
• unify access to predictive analytics (scikit-learn, R, DNN, etc) via unified JSON API and CK web services (CK repo);
• enable reproducible and interactive articles (CK repo).

Project homepage:

• http://cKnowledge.org
• http://cTuning.org

Notable use cases:

• Wikipedia article

CK concepts by Michel Steuwer:

• https://michel-steuwer.github.io/About-CK

• Permissive 3-clause BSD license. (See LICENSE.txt for more details).

The minimal installation requires:

• Python 2.7 or 3.3+ (limitation is mainly due to unitests)
• Python PIP (if you would like to install CK via PIP)
• Git command line client.

On Ubuntu, you can install these dependencies via

$ apt-get install -y python python-pip git

On Windows, you can download and install these tools from the following sites:

• Git: https://git-for-windows.github.io
• Minimal Python: https://www.python.org/downloads/windows
• Anaconda scientific Python with all packages: https://www.continuum.io/downloads#_windows

You can now install stable CK version via PIP simply as following (you may need to prefix it with "sudo" on Linux):

$ pip install ck

Alternatively, you can install development CK version in your local user space via GIT as following:

 $ git clone https://github.com/ctuning/ck.git ck

and then add CK to PATH on Linux as following:

 $ export PATH=$PWD/ck/bin:$PATH

or on Windows as following:

 $ set PATH={CURRENT PATH}\ck\bin;%PATH%

See cKnowledge.org/ai, unified CK AI API demo, Caffe/TensorFlow optimization statistics from various devices for more details.

You can now test CK by pulling and executing one of many shared workflows.

For example, you can try the following community project to unify benchmarking across ever changing hardware and software - just check that you have at least one compiler installed for your target platform as described here:

 $ ck pull repo:ck-autotuning
 $ ck pull repo:ctuning-programs

You can see shared programs in the CK format (JSON meta information describing how to compile and run shared program with all dependencies and data sets):

 $ ck list program

You can now compile a given program simply as following:

 $ ck compile program:cbench-automotive-susan --speed

Note, that CK will detect all available versions of required compilers (GCC, LLVM, ICC ...) and libraries on your system using CK customizable cross-platform package and environment manager with JSON API and meta. If some software dependencies are missing, CK will automatically install required packages as described in this article about portable CK workflows.

Now you can run your program simply as following:

 $ ck run program:cbench-automotive-susan

CK will ask you which command line to use and will automatically detect or install all required data sets.

If you have Android NDK/SDK installed on your host machine and Android device connected via ADB, you can simply compile and run the same problem on this device as following:

 $ ck compile program:cbench-automotive-susan --speed --target_os=android21-arm-v7a
 $ ck run program:cbench-automotive-susan --target_os=android21-arm-v7a

You can also participate in machine learning based compiler optimization using this CK repository and online demo.

Collective Knowledge can help you convert your ad-hoc experimental scripts to portable, customizable and reusable with automatic cross-platform software installation and web-based experimental dashboards for Artifact Evaluation at the leading conferences and journals as briefly described here. Feel free to ask for help from the CK community here!

See example of a CGO'17 distinguished artifact from the University of Cambridge implemented using CK:

• CK artifacts at GitHub
• Paper with AE appendix and CK workflow
• PDF snapshot of the interactive CK dashboard
• CK concepts

Note that you can install above repositories and dependencies to a new CK space without mixing up your current installations (similar to virtual environment) as following (substitute "export" with "set" on Windows):

 $ export CK_REPOS={path to a new CK space with all repos and dependencies}
 $ ck setup kernel --var.install_to_env=yes
 $ ck pull repo --url={URL of the artifact repository}

When CK kernels variable "install_to_env" is set to "yes", CK will be installing required packages inside CK space. In such case, when you stop evaluating artifacts, you can easily remove the whole CK space together with all installed packages.

You can check our shared workflow to crowdsource experiments across diverse software and hardware provided by volunteers aka SETI@HOME for computer systems' research:

 $ ck pull repo:ck-crowdtuning
 $ ck crowdsource experiment

For example, you can execute shared workflow for collaborative program optimization with all related artifacts, and start participating in multi-objective crowdtuning simply as following:

 $ ck crowdtune program

You can also crowd-tune GCC on Windows as following:

 $ ck crowdtune program --gcc --target_os=mingw-64

If you have GCC or LLVM compilers installed, you can start continuously crowd-tune their optimization heuristics in a quiet mode (for example overnight) via

$ ck crowdtune program --llvm --quiet
$ ck crowdtune program --gcc --quiet

This experimental workflow will be optimizing different shared workloads for multiple objectives (execution time, code size, energy, compilation time, etc) using all exposed design and optimization knobs, while sending best performing optimizations to the public CK-based server:

• http://cTuning.org/crowd-results
• http://cknowledge.org/interactive-report

CK server will, in turn, perform on-line learning to classify optimization versus workloads which can be useful for compiler/hardware designers and performance engineers (described in more detail in http://arxiv.org/abs/1506.06256 ).

You can even participate in collaborative experiments using your Android mobile phone by installing the following application from the Google Play Store:

• https://play.google.com/store/apps/details?id=openscience.crowdsource.experiments

If you want to try something more complex, you can check our cross-platform DNN unification, customized installation and benchmarking via CK with JSON API and web services:

• http://cKnowledge.org/ai

You can find already shared artifacts and repositories here:

• List of shared repositories: https://github.com/ctuning/ck/wiki/Shared-repos
• List of shared modules: https://github.com/ctuning/ck/wiki/Shared-modules

Please check out CK getting started guides and CK wiki for further details:

• https://github.com/ctuning/ck/wiki/Getting-started-guide
• https://github.com/ctuning/ck/wiki/Portable-workflows
• https://github.com/ctuning/ck/wiki

You can also reuse and customize artifacts with CK-based experimental workflows shared at the leading computer systems' conferences:

• http://cTuning.org/ae/artifacts.html

If you would like to try CK without installing it, you can run the following Docker image:

 $ docker run -it ctuning/ck

However, the idea of CK is to be able to rebuild user experimental workflows natively to take advantage of the latest software environment.

Also note that we added Docker automation to CK (to help evaluate artifacts at the conferences, share interactive and reproducible articles or crowdsource experiments for example).

Please check 'ck-docker' repository at GitHub:

 $ ck show repo:ck-docker

You can download and view one of our CK-based interactive and reproducible articles as following:

 $ ck pull repo:ck-docker
 $ ck run docker:ck-interactive-article --browser (--sudo)

You can test CK functionality via

 $ ck run test

• Artifact Evaluation for computer systems' conferences: http://cTuning.org/ae
• New publication model with the community-driven public reviewing: http://adapt-workshop.org

• https://github.com/ctuning/ck/wiki/Plans

• https://github.com/ctuning/ck/wiki/Motivation

• Grigori Fursin, cTuning foundation / dividiti
• Anton Lokhmotov, dividiti

• Public wiki with open challenges in computer engineering: https://github.com/ctuning/ck/wiki/Research-and-development-challenges
• Mailing list for open, collaborative and reproducible R&D including knowledge preservation, sharing and reuse: http://groups.google.com/group/collective-knowledge
• Mailing list for software and hardware multi-objective (performance/energy/accuracy/size/reliability/cost) benchmarking, autotuning, crowdtuning and run-time adaptation: http://groups.google.com/group/ctuning-discussions

@inproceedings{ck-date16,
    title = {{Collective Knowledge}: towards {R\&D} sustainability},
    author = {Fursin, Grigori and Lokhmotov, Anton and Plowman, Ed},
    booktitle = {Proceedings of the Conference on Design, Automation and Test in Europe (DATE'16)},
    year = {2016},
    month = {March},
    url = {https://www.researchgate.net/publication/304010295_Collective_Knowledge_Towards_RD_Sustainability}
}

The concepts have been described in the following publications:

• http://tinyurl.com/zyupd5v (DATE'16)

• http://arxiv.org/abs/1506.06256 (CPC'15)

• http://hal.inria.fr/hal-01054763 (Journal of Scientific Programming'14)

• http://bit.ly/ck-multiprog16 (MULTIPROG'16)

• http://cknowledge.org/interactive-report

• http://arxiv.org/abs/1406.4020 (TRUST'14 @ PLDI'14)

• https://hal.inria.fr/inria-00436029 (GCC Summit'09)

If you found CK useful, feel free to reference any of the above publications. You can download all above references in one BibTex file here.

• 2016: General Motors and dividiti use CK to crowdsource benchmarking and optimization of CAFFE: public CK repo
• 2015: ARM and the cTuning foundation use CK to accelerate computer engineering: HiPEAC Info'45 page 17, ARM TechCon'16 presentation and demo, public CK repo
• 2014: HiPEAC technology transfer award: HiPEAC TT winners

CK development is coordinated by the cTuning foundation (non-profit research organization). We thank the EU TETRACOM 609491 Coordination Action for initial funding and dividiti for continuing support. We would also like to thank Microsoft Research program for one-year grant to host the CK public repository in the Azure cloud. We are also extremely grateful to all volunteers for their valuable feedback and contributions.