S3 identifier with `@` in key raises RuntimeError about smart_open HOT 7 CLOSED

The name doesn't follow AWS best practices. What is the reason for using it?

The "@" symbol is not considered a safe character in AWS guide. It is referenced as "character that requires special handling".

from smart_open.

Need to add logic to distinguish s3://key:secret@bucket/object vs s3://bucket/obj@ect and s3://bucket/ob:j@ect

from smart_open.

Is it possible to do so cleanly & unambiguously?

Is the S3 URI syntax described somewhere formally, so we know for sure?

from smart_open.

Maybe if slashes / are not allowed in credentials (I'm not sure -- maybe they are), a simple check would be "credentials before @ contain a / => they're not really credentials, treat @ as part of filename".

from smart_open.

/ is allowed in the credentials - secret keys usually have a slash in them

I think these are the two additional tests we need -

        # correct uri, key contains @
        parsed_uri = smart_open.ParseUri("s3://mybucket/mykey@mydir")
        self.assertEqual(parsed_uri.scheme, "s3")
        self.assertEqual(parsed_uri.bucket_id, "mybucket")
        self.assertEqual(parsed_uri.key_id, "mykey@mydir")
        self.assertEqual(parsed_uri.access_id, None)
        self.assertEqual(parsed_uri.access_secret, None)

        # correct uri with credentials, key contains @
        parsed_uri = smart_open.ParseUri("s3://ACCESSID456:acces/sse_cr-et@mybucket/mykey@mydir")
        self.assertEqual(parsed_uri.scheme, "s3")
        self.assertEqual(parsed_uri.bucket_id, "mybucket")
        self.assertEqual(parsed_uri.key_id, "mykey@mydir")
        self.assertEqual(parsed_uri.access_id, "ACCESSID456")
        self.assertEqual(parsed_uri.access_secret, "acces/sse_cr-et")

Seems slightly difficult to do cleanly, lots of edge cases.

from smart_open.

The Access Key ID is 20 alpha-numeric characters, so a regex can locate the s3://key:secret@bucket/object. Will send a fix later today.

from smart_open.

The problem still exists

from smart_open.