[Bug] config to import yapf causes 'EOFError: Ran out of input' when distributed training about mmengine HOT 9 CLOSED

Same problem using the older mmcv-full (1.3.0) and mmseg=0.11.0.
Using yapf=0.40.1

Small update: pip install yapf=0.32 fixes this issue for me.

from mmengine.

I used a wait function to control each rank's import, and make sure they are imported one by one. Stupid, but working

import time
import os

def wait_before_import_config():
    t = int(os.environ.get('LOCAL_RANK', 0))
    time.sleep(t * 0.5)

def wait_after_import_config():
    t = int(os.environ.get('WORLD_SIZE', 0)) - int(os.environ.get('LOCAL_RANK', 0))
    time.sleep(t * 0.5)

wait_before_import_config()
from mmengine.config import Config
wait_after_import_config()

from mmengine.

I think it's a yapf problem, the repo of yapf also reported this link

from mmengine.

I am getting this as well

from mmengine.

In my case, setup is:

"mmengine==0.10.2"
"mmcv==2.1.0"
"mmdet==3.3.0"

on ubuntu

from mmengine.

This has been happening a lot for us. On older versions of mmcv.

from mmengine.

@DeclK you are a hero!

from mmengine.

I used a wait function to control each rank's import, and make sure they are imported one by one. Stupid, but working

import time
import os

def wait_before_import_config():
    t = int(os.environ.get('LOCAL_RANK', 0))
    time.sleep(t * 0.5)

def wait_after_import_config():
    t = int(os.environ.get('WORLD_SIZE', 0)) - int(os.environ.get('LOCAL_RANK', 0))
    time.sleep(t * 0.5)

wait_before_import_config()
from mmengine.config import Config
wait_after_import_config()

pls check this issue,may be helpful for you:google/yapf#1204

from mmengine.

Similar problem i meet when using mmengine,how i find a way to fix in this issue:google/yapf#1204

from mmengine.