hardcoded relative paths about firestorm HOT 6 CLOSED

@altaetran currently, the classpath for hadoop related is defined in start-coordinator.sh & start-shuffle-server.sh
I think there should be a guide to help user update the path with different environment.

from firestorm.

Hi Colin, Thanks for the response. I was able to change the paths explicitly to the corresponding absolute paths. I was able to get the start-coordinator.sh to run, but I am not ure if the shuffle server was able to run, as it ran normally, but unlike with the start-coordinator.sh, I was able to run the start-shuffle-server.sh multiple times without a warning about a process already being started. Moreover, upon trying to run in spark, normal map only tasks worked properly, but when I tried to do a shuffle task, the entire process started to hang without any progress (task completion bar did not even show up). Do you know if there is a way to verify if the start-shuffle-server.sh ran properly? Thanks!

from firestorm.

@altaetran There has RestApi in both coordinator & shuffle server, you can check it to confirm if shuffle server do the heartbeat and available. The url should be http://coordinator:<rss.jetty.http.port>/metrics/server.
You can also check if shuffle server works as expected by RestApi.
I think document should be updated according to all problems from users, and I'll update it soon, maybe this week.

from firestorm.

Firestorm is now moved to ASF and renamed to Apache Uniffle (incubating).
This issue is solved by apache/incubator-uniffle#29

from firestorm.

from firestorm.

Hi @altaetran Firestorm project has already moved to Apache Uniffle (https://github.com/apache/incubator-uniffle), would you please create a new issue or discussion in the new repo? Thanks.

from firestorm.