Unable to Connect to HDFS for Tiered Storage Setup about clickhouse HOT 7 OPEN

HDFS is unusable as a MergeTree disk with the current Clickhouse implementation

• #43112

from clickhouse.

Hey @den-crane,
Thank you for the quick response.

After reviewing the mentioned issue, it seems ClickHouse doesn't support HDFS. Is it possible to use a mounted or external disk for 'cold' storage instead?

Any references or documentation on this would be greatly appreciated.
Thank you!

from clickhouse.

it seems ClickHouse doesn't support HDFS.

right.

Is it possible to use a mounted or external disk for 'cold' storage instead?

HDFS as an mounted/external disk? Never heard about such technology, I doubt it will work.

from clickhouse.

Hey @den-crane

No not HDFS as mounted disk.
Can i just use any storage disk like HDD or SSD as cold storage, which is mounted to the Clickhouse server machine ?

Thank you!

from clickhouse.

Can i just use any storage disk like HDD or SSD as cold storage, which is mounted to the Clickhouse server machine ?

Yes, you can.

from clickhouse.

Hey @den-crane,

I'm trying to use an external disk mounted to the ClickHouse server machine for cold storage, but I'm encountering the following error:

<Error> Application: std::exception. Code: 1001, type: std::__1::__fs::filesystem::filesystem_error, e.what() = filesystem error: in posix_stat: failed to determine attributes for the specified path: Permission denied [/home/server-user/cold-storage/]

I have verified that the specified path /home/server-user/cold-storage/ has the correct permissions set for server-user. However, when ClickHouse attempts to write data to this path, it throws the above error.

Additional Information:

• The path does not have root permissions, it is owned by server-user.
• I have come across several issues suggesting downgrading ClickHouse to version 22.x.x as a potential solution.

Could you please advise on the best approach to resolve this issue? Is downgrading ClickHouse a viable solution, or is there another recommended method to address this permission error?

Thank you!

from clickhouse.

@avj-vaibhav

1. how do you run Clickhouse-server?
2. check sudo -u server-user 'mkdir /home/server-user/cold-storage/test'
3. what is this /cold-storage/ sortof USB disk? what filesystem does it have?

from clickhouse.