imageworks / pystring Goto Github PK

On Katana style scenegraph paths, no less! :(

6e84cef

My little app crashes with segfault, here's Valgrind diagnosis (after using pystring::split in code):

6,253 bytes in 117 blocks are possibly lost in loss record 13 of 14
  in pystring::(anonymous namespace)::split_whitespace(std::string const&, std::vector<std::string, std::allocator<std::string> >&, int) in pystring/pystring.cpp:109
  1: operator new(unsigned long) in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so
  2: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) in /usr/lib/libstdc++.so.6.0.18
  3: char* std::string::_S_construct<char*>(char*, char*, std::allocator<char> const&, std::forward_iterator_tag) in /usr/lib/libstdc++.so.6.0.18
  4: std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&, unsigned long, unsigned long) in /usr/lib/libstdc++.so.6.0.18
  5: std::string::substr(unsigned long, unsigned long) const in /usr/lib/libstdc++.so.6.0.18
  6: pystring::(anonymous namespace)::split_whitespace(std::string const&, std::vector<std::string, std::allocator<std::string> >&, int) in pystring/pystring.cpp:109
  7: pystring::split(std::string const&, std::vector<std::string, std::allocator<std::string> >&, std::string const&, int) in pystring/pystring.cpp:172
  8: ProcessLine(std::string const&, datastorage&) in main.cpp:54
  9: ProcessFile(char const*) in main.cpp:41
  10: main in main.cpp:28

See 53e5b88

pystring::find("", "") -> -1
    Correct: 0
pystring::find("abcabcabc", "bc", 4, 5) -> 4
    Correct: -1

http://code.google.com/p/pystring/issues/detail?id=2

Reported by [email protected], Jul 19, 2010

What steps will reproduce the problem?

1. strip("")

What is the expected output? What do you see instead?
The expected output is empty string. Instead the function crashes.

What version of the product are you using? On what operating system?
My version is from pystring_snapshot_012810.zip. Operating system is Windows XP, compiler is Visual Studio 2008.

Comment 1 by project member [email protected], Feb 8, 2011
The Python strip function returns the original string unaltered when the strip characters is empty.

rsplit makes use of std::string::size_type for internal counters, but also can decrement them in some cases. we need to confirm that it's not more appropriate to be using ssize_t for these.

See a5c3cfa for another example of this bug

Return a copy of the string, concatenated N times, together. Corresponds to the mul operator.

See opencolorio/pystring for an implementation.

http://code.google.com/p/pystring/issues/detail?id=3

Reported by [email protected], Jul 26, 2010
What steps will reproduce the problem?

1. Call endwith function with suffix size > str size, e.g. pystring::endswith("help", ".mesh") under Visual Studio 2008
2. There is an overflow of "lower" variable inside endswith source.
3. __substrcmp(str, suffix, lower ) crash

What is the expected output? What do you see instead?

There should be correct handling of this situation: str size < suffix size.

What version of the product are you using? On what operating system?

I don't know pystring version. I use Visual Studio 2008 Express. But this bug does not reproduce on GCC4.

Please provide any additional information below.

Just do a simple demo: pystring::endswith("help", ".mesh"); under Visual Studio 2008.

Comment 1 by project member [email protected], Feb 8, 2011
I can confirm this error. The internal substrcmp function is being called with a negative 'lower' position argument. Should be easy to add a comparison for this case. Need to investigate if 'startswith' suffers a similar problem. Also need to do more testing considering each of these functions take an optional 'start' and 'end' argument.

With what lowest version of the C++ standard is pystring expected to work?

Is there a strong reason why the current API shies away from ever returning a vector of strings as a direct return value, like the actual Python API?

This is particularly useful with C++11 implementations, where auto segs = split(s, delim) is much more concise than having to make a separate and fully-qualified declaration for the std::vector<std::string> segs object before passing it as a mutable reference. In my understanding, modern compilers have no trouble optimizing away this pattern into an equivalent in-place mutation of the return object.

std::string s = "HKEY_CURRENT_USER\\Software\\AppDataLow\\gogo";
std::cout << pystring::lstrip(s, "HKEY_CURRENT_USER\\") << std::endl;
// oftware\AppDataLow\gogo

the S disappear

Hello
I have build issue
http://dpaste.com/5CCL3ENUA
Can you help me?

This isn't really a show stopper, but on Windows the tests don't agree with the behavior, so the abspath tests fail. The actual functionality seems sane and intentional, so I dunno if the fix is to change the behavior of the implementation to be consistent across platforms, or probably just change the expectation of the tests on Windows (or skip the abspath tests on Windows entirely).

Anyhow, here's what it is doing:
In test.cpp:
PYSTRING_CHECK_EQUAL(pystring::os::path::abspath("", "/net"), "/net");

But on windows the implementation uses "" in canonical names rather than "/", because Windows. So it fails because "\net" isn't equal to "/net"
The implementation of abspath on Windows runs return normpath_nt(p); to convert the '/' to '' here
https://github.com/wrosecrans/pystring/blob/build/pystring.cpp#L1456

Here is a snippet of the output from running the tests:

Test [pystring] [translate] - PASSED
D:\a\1\s\test.cpp:516:
FAILED: pystring::os::path::abspath("", "/net") == "/net"
values were '\net' and '/net'
D:\a\1\s\test.cpp:517:
FAILED: pystring::os::path::abspath("../jeremys", "/net/soft_scratch/users/stevel") == "/net/soft_scratch/users/jeremys"
values were '\net\soft_scratch\users\jeremys' and '/net/soft_scratch/users/jeremys'
D:\a\1\s\test.cpp:518:
FAILED: pystring::os::path::abspath("../../../../tmp/a", "/net/soft_scratch/users/stevel") == "/tmp/a"
values were '\tmp\a' and '/tmp/a'
Test [pystring] [abspath] - FAILED
Test [pystring_os_path] [splitdrive] - PASSED
Test [pystring_os_path] [isabs] - PASSED

In the context of doing CI for a thing that uses PyString, it's convenient to be able to use the tests as a sanity check to make sure I haven't screwed up something completely in my build setup. I am using CMake to build with the MSVC compiler instead of the Makefile that uses G++, but it's not doing anything particularly exciting:

https://github.com/wrosecrans/pystring/blob/build/CMakeLists.txt

Functions like pystring::os::path::isabs_posix call pystring::startswith:

bool isabs_posix(const std::string & s) { return pystring::startswith(s, "/"); }

And it seems that this will create a new std::string for "/" each time this is called which will dominate the execution time of code that calls isabs() repeatedly. Would it be reasonable to declare some of these constants to amortize this cost?

bool isabs_posix(const std::string & s) { static const std::string slash("/"); return pystring::startswith(s, slash); }

It seems "/" and "\" is used several times in pystring as the arguments to startswith, endswith, strip, etc. which take references to std::string as arguments and this could be more optimal for the cases where these functions get called a lot?

Last release tag in this repository is v1.1.3 (Oct 17, 2012), however there are more recent commits.
Which one do you recommended? The 1.1.3 tagged release or cloning the latest commit?
Thank you!

http://code.google.com/p/pystring/issues/detail?id=4

Reported by [email protected], Dec 21, 2010
in line 765
s += table[ s[i] ];
should be:
s += table[ str[i] ];

This line deadlocks if path has double slash, like in "D:/dir//"

I think the fix is to slice head2, not head:
head2 = pystring::slice(head2,0,-1);

Consider supporting std::wstring

Sure, the python string functions are awesome if you need them in C++, but what is one to do if you need them in python? Perhaps boost python would be appropriate? (Please ping the developers before attempting this task, though). ;)

Hi,

The license of this project seems to match the overall license of OpenColorIO (the only difference is that it mentions "Sony" directly, instead of referring only to the copyright holders.

It would help to have it explicitly detected as [BSD 3-Clause "New" or "Revised" License] by github.

More context, I'm updating Blender's 3rd party license document for Blender 3.2. We now depend on Pystrings since OCIO itselfs depends on it during build time.

It helps a lot to quickly find what is the license of a dependency without having to compare the license line-by-line to see if it is compatible with GPL :)

Running "make install" installs libpystring.so.0.0.0 and libpystring.so.0 when the version on the released file is 1.1.3 . Also pystring.h is not installed.