Generate events or process miniAOD files with CRAB3.
First, get this repository:
git clone https://github.com/IzaakWN/CRAB CRAB
cd CRAB
then install the relevant CMSSW in this directory, e.g.:
CMSSW=CMSSW_10_6_19
export SCRAM_ARCH=slc7_amd64_gcc700
cmsrel $CMSSW
cd $CMSSW/src
cmsenv
scram b -j4
cd ../..
You can find a list of supported CMSSW versions and architectures on this page.
When you want to use CRAB, you need to source the relevant setup script:
source $VO_CMS_SW_DIR/crab3/crab_slc6.sh # for SLC6
source $VO_CMS_SW_DIR/crab3/crab.sh # for SLC7
Make sure you have a valid VOMS proxy,
voms-proxy-info --timeleft # check how many seconds you have left
voms-proxy-init -voms cms -valid 400:0 # renew proxy if it is too short
source setupVOMS.sh # OR, source this script instead of the above two lines
Before submitting jobs to CRAB, make sure that in submit_crab.py you have specified the correct tier storage element that you have writing permissions to. For example with 'T2_CH_CSCS' for PSI's T2:
sed "s/\(site\s*=\s*\)'\w+'/\1'T2_CH_CSCS'/" submit_crab.py
In case you want to use DeepTau2017v2p1 in 102X samples, see this this TWiki page, and setup CMSSW as
CMSSW=CMSSW_10_2_16_patch1
export SCRAM_ARCH=slc6_amd64_gcc700
cmsrel $CMSSW
cd $CMSSW/src/
cmsenv
git cms-merge-topic -u cms-tau-pog:CMSSW_10_2_X_tau-pog_DeepTau2017v2
git cms-merge-topic -u cms-tau-pog:CMSSW_10_2_X_tau-pog_deepTauVetoPCA
sed 's/idDeepTau2017v2/idDeepTau2017v2p1/g' PhysicsTools/NanoAOD/python/taus_cff.py -i
sed 's/rawDeepTau2017v2/rawDeepTau2017v2p1/g' PhysicsTools/NanoAOD/python/taus_cff.py -i
scram b -j 4
cd ../..
With each new shell session, do something like
cd CRAB
source setupVOMS.sh
source $VO_CMS_SW_DIR/cmsset_default.sh
source $VO_CMS_SW_DIR/crab3/crab_slc6.sh
export SCRAM_ARCH=slc7_amd64_gcc700
cd CMSSW_10_6_19/src
cmsenv
cd ../..
Get some files with e.g.
dasgoclient --limit=0 --query="dataset=/DYJetsToLL_M-50_TuneCP5_13TeV-madgraphMLM-pythia8/RunIIAutumn18MiniAOD-102X_upgrade2018_realistic_v15-v1/MINIAODSIM file" | head -n1
xrdcp -f root://cms-xrd-global.cern.ch//store/mc/RunIIAutumn18MiniAOD/DYJetsToLL_M-50_TuneCP5_13TeV-madgraphMLM-pythia8/MINIAODSIM/102X_upgrade2018_realistic_v15-v1/100000/042C8EE9-9431-5443-88C8-77F1D910B3A5.root ./test.root
Reprocess miniAOD to add DeepTau2017v2p1 with
cmsRun pset_miniAOD_rerun.py
Process miniAOD to nanoAODv5 with
cmsRun pset_nanoAODv5.py
Check the content of a miniAOD file with
edmProvDump input/VLQ-p_M1100_2017_rerun.root > dump.log
or use as an example (see this TWiki page)
./checkMiniAOD.py
Collect your favorite samples in the respective python file, sorted in a dictionary per year:
samples_miniAOD.py
samples_nanoAOD.py
Then you can submit a specific parameter-set configuration file as
./submit_crab.py pset_nanoAODv5.py -y 2017
To filter a specific sample, do e.g.
./submit_crab.py -y 2017 -s DY*Jets
To run test job(s), do e.g.
./submit_crab.py -y 2017 -t # one test job
./submit_crab.py -y 2017 -t 2 # two test jobs
Check the task status, and resubmit if needed as:
crab status -d crab_projects/crab_<request>
crab resubmit -d crab_projects/crab_<request>
replacing <request>. Or, use the local crab.sh script to check many directories at once
crab.sh status crab_projects/crab_*
crab.sh resubmit crab_projects/crab_*
If you publish your output files (default, if it is not a test job), you can retrieve them in DAS with (replacing <user>)
dasgoclient --limit=0 --query="dataset=/*/<user>*/USER instance=prod/phys03"
Before you can generate MadGraph events with CMSSW, it's useful to create a gridpack, as described here, or install
cd $CMSSW_BASE/src/
git clone [email protected]:cms-sw/genproductions.git genproductions
and in a clean shell session (i.e., without an CMSSW environment sourced), run
source $VO_CMS_SW_DIR/cmsset_default.sh
cd genproductions/bin/MadGraph5_aMCatNLO/
./gridpack_generation.sh <process name without _proc_card.dat> <card dir>
To produce a large set of MadGraph cards with varying parameters (mass, coupling strenghts, ...), you can prepare some template cards like the examples in cards/ScalarLQ_Single/ScalarLQ_Single_template_*.dat, and use create_cards.py as e.g.
./create_cards.py cards/ScalarLQ_Single/ScalarLQ_Single_template_*.dat -m 1000 -p LAMBDA=0.1,1.0,1.5
Produce the cards and gridpacks in series with generate_gridpacks.py, e.g.
./generate_gridpacks.py cards/ScalarLQ_Single ScalarLQ_Single -m 1000 -p LAMBDA=0.1,1.0,1.5 -C CMSSW_10_6_19
Produce events with a gridpack with pset_GENSIM.py as e.g.
cmsRun pset_GENSIM.py nevents=100 gridpack=ScalarLQ_Single_M500_slc6_amd64_gcc630_CMSSW_9_3_16_tarball.tar.xz
To submit a MCM fragment and produce GENSIM events on HTCondor for quick studies, do e.g.
mkdir -p test_DYJetsToMuTauh && cd test_DYJetsToMuTauh
curl -s -k https://cms-pdmv.cern.ch/mcm/public/restapi/requests/get_test/TAU-RunIISummer20UL18wmLHEGEN-00006 > setup_DYJetsToMuTauh.sh
sed -i 's/exit $GEN_ERR/echo "# IGNORE ERROR $GEN_ERR from request_fragment_check.py"/' setup_DYJetsToMuTauh.sh
sed -i 's/EVENTS=[0-9]\+/EVENTS=500/' setup_DYJetsToMuTauh.sh
bash setup_DYJetsToMuTauh.sh # setup CMSSW, fragment and generate GENSIM
./submit_fragment.py -t _DYJetsToMuTauh test_DYJetsToMuTauh/TAU-RunIISummer20UL18wmLHEGEN-00006_1_cfg.py -N 100 -n 10000 -v2
Instructions to get the fragment are below.
- Tutorial: https://twiki.cern.ch/twiki/bin/view/CMSPublic/WorkBookCRAB3Tutorial
- Configuration: https://twiki.cern.ch/twiki/bin/view/CMSPublic/CRAB3ConfigurationFile
- Commands: https://twiki.cern.ch/twiki/bin/view/CMSPublic/CRAB3Commands
- FAQ: https://twiki.cern.ch/twiki/bin/view/CMSPublic/CRAB3FAQ
- Dashboard: https://monit-grafana.cern.ch (Jobs > CMS Tasks Monitoring GlobalView > Select User >
<username>)
- working book: https://twiki.cern.ch/twiki/bin/view/CMSPublic/WorkBookNanoAOD
- 2016
9_4_X: https://cms-nanoaod-integration.web.cern.ch/integration/master/mc94X2016_doc.html - 2017
9_4_X: https://cms-nanoaod-integration.web.cern.ch/integration/master/mc94X_doc.html - 2018
10_2_X: https://cms-nanoaod-integration.web.cern.ch/integration/master-102X/mc102X_doc.html
More notes.
To get a fragment of a PSet configuration file with cmsDriver.py is explained in the NanoAOD Workbook.
cmsDriver.py myNanoProdMc -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --no_exec --conditions MyGlobalTag --era MyEraModifiers
cmsDriver.py myNanoProdData -s NANO --data --eventcontent NANOAOD --datatier NANOAOD --no_exec --conditions MyGlobalTag --era MyEraModifiers
Get a fragment of a PSet configuration file for a particular sample:
- Go to MCM.
- Go to 'Request' > 'Output Dataset', and type in the DAS path of your favorite sample, e.g.
/DYJetsToLL_M-50_TuneCP5_13TeV-madgraphMLM-pythia8/RunIIAutumn18MiniAOD-102X_upgrade2018_realistic_v15-v1/MINIAODSIM
- In the 'Actions' column, click on the 'Get setup command' symbol (circle with down arrow).
https://cms-pdmv.cern.ch/mcm/public/restapi/requests/get_setup/MUO-RunIIAutumn18MiniAOD-00016
https://cms-pdmv.cern.ch/mcm/public/restapi/requests/get_fragment/MUO-RunIIAutumn18MiniAOD-00016 - Copy-paste the
cmsDriver.pythe command line (in aCMSSWenvironment). - You will find a configuration file called
<prep-id>.py.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent