ccurtsinger / stabilizer Goto Github PK
View Code? Open in Web Editor NEWStabilizer: Rigorous Performance Evaluation
Home Page: http://github.com/ccurtsinger/stabilizer
License: Apache License 2.0
Stabilizer: Rigorous Performance Evaluation
Home Page: http://github.com/ccurtsinger/stabilizer
License: Apache License 2.0
Hi,
I've tried to run stabilizer in a container to be able to use it as-is in a deprecated environment (llvm 3.1) for small files. However, it doesn't build, and my cmake knowledge is very limited. I'll dig more into that later, but if you have any pointer, I'd be thankful!
FROM alpine:3.9
RUN apk add --no-cache git cmake build-base python \
&& rm -rf /tmp/* /var/cache/apk/*
RUN git clone --depth 1 --branch llvmorg-3.1.0 https://github.com/llvm/llvm-project.git
RUN cd llvm-project \
&& mkdir build \
&& cd build \
&& cmake -DLLVM_ENABLE_PROJECTS=clang -G "Unix Makefiles" ../llvm \
&& make
RUN git clone git://github.com/ccurtsinger/stabilizer.git stabilizer
RUN cd stabilizer && make
Currently the bug seems to be that I need to specify another std version to CMAKE, yet I've tried many way of doing so without success. The cmake
command works, and the make
command fails at:
[ 5%] Building CXX object lib/Support/CMakeFiles/LLVMSupport.dir/DynamicLibrary.cpp.o
/llvm-project/llvm/lib/Support/DynamicLibrary.cpp: In static member function 'static void* llvm::sys::DynamicLibrary::SearchForAddressOfSymbol(const char*)':
/llvm-project/llvm/lib/Support/DynamicLibrary.cpp:158:42: error: invalid conversion from 'const void*' to 'void*' [-fpermissive]
if (!strcmp(symbolName, #SYM)) return &SYM
^
/llvm-project/llvm/lib/Support/DynamicLibrary.cpp:165:5: note: in expansion of macro 'EXPLICIT_SYMBOL'
EXPLICIT_SYMBOL(stderr);
^~~~~~~~~~~~~~~
/llvm-project/llvm/lib/Support/DynamicLibrary.cpp:158:42: error: invalid conversion from 'const void*' to 'void*' [-fpermissive]
if (!strcmp(symbolName, #SYM)) return &SYM
^
/llvm-project/llvm/lib/Support/DynamicLibrary.cpp:166:5: note: in expansion of macro 'EXPLICIT_SYMBOL'
EXPLICIT_SYMBOL(stdout);
^~~~~~~~~~~~~~~
/llvm-project/llvm/lib/Support/DynamicLibrary.cpp:158:42: error: invalid conversion from 'const void*' to 'void*' [-fpermissive]
if (!strcmp(symbolName, #SYM)) return &SYM
^
/llvm-project/llvm/lib/Support/DynamicLibrary.cpp:167:5: note: in expansion of macro 'EXPLICIT_SYMBOL'
EXPLICIT_SYMBOL(stdin);
^~~~~~~~~~~~~~~
make[2]: *** [lib/Support/CMakeFiles/LLVMSupport.dir/build.make:830: lib/Support/CMakeFiles/LLVMSupport.dir/DynamicLibrary.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:91: lib/Support/CMakeFiles/LLVMSupport.dir/all] Error 2
make: *** [Makefile:152: all] Error 2
Ulysse
Hi all,
As stabilizer is not compatible with LLVM versions that still works on current OSes (we're 7 versions of LLVM ahead now, no C++17, etc), is there a simpler way to approach the effect of Stabilizer? In "Producing Wrong Data Without Doing Anything Obviously Wrong!", they randomized the order of the .o file, and change the environment size. Is that still the second-best option?
It's not strictly linked to stabilizer, but I'm not sure where to ask that and it's a discussion which is way too much forgotten in computer sciences...
Thanks,
Tom
Seems the Paper link in the readme broke.
I would really appreciate if the STABILIZER would support ARM64.
Maybe you could take this into consideration.
Best
people are asking.
Hello,
I'm wondering if I can get stabilizer to work with cargo
/ rustc
?
If you're not familiar, rustc
uses LLVM as a backend.
Looks like a very useful tool.
Thank you.
Hey @ccurtsinger,
Just listened to the presentation of Emery.
Now I would love to use it. Using it would make much more sence and reduce variance cause in many cases. (currently measuring tests with cargo bench, so not that great). Now yes, I read the readme, it says that the project isn't maintained. May I ask why?
Is there a plan/movement going for something new? did a miss an alternative? Would really love to see that thing fly and sure, if theres something to do or helpout I'm in.
Cheers ๐ป
Yes, this is a little shameless self-promotion, but hopefully for the good of the project.
Because this project has not been maintained for years, and there is a fragmented fork network with mostly half-working versions of Stabilizer, I have taken it upon myself to fork this project and will try to actively maintain it. Note: @ccurtsinger has not endorsed this in any way.
I have made some changes in my fork however.
One of the biggest hurdles for anyone wanting to test Stabilizer with their own application is that szc
will fail even the most basic CMake or AutoConf compiler detection. Therefore, before anyone can even test this they might have to do a big rewrite of their applications build system, that would be very hard for bigger projects like Chromium for example.
Therefore I have rewritten the compiler wrapper from scratch, and renamed it to szcc
. Now most compiler detection code will succeed, and can it often be used as a drop-in compiler "replacement" for many applications. There are still corner-case bugs, for example I know that gbench fails to compile, and I will accept PRs to fix these if you have the interest and time to contribute.
This makes it a lot easier for anyone to test Stabilizer without investing a lot of time and effort into it before even being able to do anything basic with it.
Unfortunately although Stabilizer has been attempted ported to newer versions of LLVM, and compiles with those, it is causing crashes. Probably the porting effort has been best-effort and is just incomplete. Cudos to @Fusilet @dendibakh @jgall @magras @timadye and others for their efforts though.
One major problem for me is that I do not know how LLVM-pass stuff works at all, so while I have been looking at that code to attempt to find the culprits of the Code Randomization
and Stack Randomization
crashes, I have very little idea what I'm even looking for. One of those issues has been documented in Issue 1 on my fork so that anyone interested can read up on the symptoms, if anyone has anything to contribute to that analysis or even better a bugfix then that is greatly appreciated.
Hopefully now that Stabilizer is easier to use, it will also be easier to find someone interested in contributing fixes.
In other words, I will attempt to actively maintain the repo and fix any issues that I am able to, but I will also need help to fix some of the Stabilizer functionality. I intentionally broke the Github fork-tree so that it will be less confusing and easier to find up-to-date forks, this Github feature is really not great when the parent project has gone dormant.
If @ccurtsinger wants to become an active maintainer of Stabilizer again, I will happily step aside. My main interest and motivation is that this amazing tool becomes more useful to all the open-source projects out there. If @ccurtsinger would contribute some insight into what the crash bugs are likely caused by, that would be really awesome of course ๐.
Please have a look at the readme of my fork for more information about the changes I have made https://github.com/Dead2/stabilizer
Dear @ccurtsinger ,
We are now working on a llvm course project, and we are required to use stabilizer to analyze performance.
However, our project is built based on llvm 4.0.
So is it possible to build stabilizer in llvm 4.0?
Thank you very much!
Yiyi
Hi,
It is awesome to find such an interesting paper/project for rigorous performance evaluation.
I have one question regarding to the randomization effect.
If programmer manually specifies some parts of memory layout of the program, like cache alignment for global struct, will stabilizer preserve the information set by the programmer? Does heap/code/stack randomization respect the setting?
Best
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.