Allow lists with arbitrary strings in CVS import by escaping array delimiter about neo4j HOT 5 CLOSED

Hi @nichtich, thanks for the feedback on the CSV Import feature!

Since you mentioned escaping the array delimiter by doubling it, it looks like that you have some control over the generation of the CSVs or maybe a pre-processing phase before passing them to neo4j-admin import. So it looks like you're able to also use any arbitrary array delimiter, that would be correct?

You also said that you already thought about using another array delimiter. Besides the null-byte (which does not look like a good one), are you confident that your strings would contain even the ASCII delimiter characters for example? The import command also allows you to set the array delimiter as any unicode character, so you're not restricted to ASCII either. Maybe this could help you with your current requirements?

from neo4j.

Thanks for the quick reply. Any special character (null byte, information separator code...) is a workaround with drawback of hardly being readable and editable by hand (and one must be sure to not use the character in actual data). For my current use case this is a minor issue indeed.

I'll keep the issue open because even if not being implemented, the limitation and workaround should be made clear in the documentation. The documentation may use the example --array-delimiter="\x1C" because escaping of control characters may be unknown to users (I had to look up the right way). Alternatively allow --array-delimiter=U+001C to use a specific character given by Unicode code point.

from neo4j.

Any special character (null byte, information separator code...) is a workaround with drawback of hardly being readable and editable by hand (and one must be sure to not use the character in actual data). For my current use case this is a minor issue indeed.

Handling arbitrary content (or providing the mechanism to do so) is always tricky! For example, if we support the double escaping, then it leads to what we do when we want a property array to contain an empty element (e.g. 'x||y|z' wants {name:["x", "", "y","z"]}) 🙃

I'm glad to hear that this is blocking you in using Neo4j though! The team added a note in our backlog of the wish for more flexibility in this aspect. If we revisit the property array parsing mechanism, we'll take this into consideration :)

I'll keep the issue open because even if not being implemented, the limitation and workaround should be made clear in the documentation. The documentation may use the example --array-delimiter="\x1C" because escaping of control characters may be unknown to users (I had to look up the right way).

About the documentation, this is definitely something we can expand on. To double check, are you mentioning this page or somewhere else where you feel we missed the opportunity to clarify how to use/pass control characters?

Alternatively allow --array-delimiter=U+001C to use a specific character given by Unicode code point.

--array-delimiter (like other delimiters) supports unicode code points. Have you found an issue when trying to use it?

from neo4j.

About the documentation, this is definitely something we can expand on. To double check, are you mentioning this page

Yes

--array-delimiter (like other delimiters) supports unicode code points. Have you found an issue when trying to use it?

The issue is only how to escape a Unicode code point in the shell. This is no big thing, just something that users might need to look up if it is not shown in the documentation example.

from neo4j.

Hey @nichtich, we have updated the examples to show how one can use the unicode code point for specifying a delimiter.

from neo4j.