phadej / igbinary Goto Github PK

Igbinary is a drop in replacement for the standard php serializer. Check https://github.com/igbinary/igbinary for the freshest version

License: Other

C 57.70% PHP 42.17% Shell 0.14%

igbinary's Introduction

igbinary

Igbinary is a drop in replacement for the standard php serializer. Instead of time and space consuming textual representation, igbinary stores php data structures in compact binary form. Savings are significant when using memcached or similar memory based storages for serialized data. About 50% reduction in storage requirement can be expected. Specific number depends on your data.

Unserialization performance is at least on par with the standard PHP serializer. Serialization performance depends on the "compact_strings" option which enables duplicate string tracking. String are inserted to a hash table which adds some overhead. In usual scenarios this does not have much significance since usage pattern is "serialize rarely, unserialize often". With "compact_strings" option igbinary is usually a bit slower than the standard serializer. Without it, a bit faster.

Features

Supports same data types as the standard PHP serializer: null, bool, int, float, string, array and objects.
__autoload & unserialize_callback_func
__sleep & __wakeup
Serializable -interface
Data portability between platforms (32/64bit, endianess)
Tested on Linux amd64, Linux ARM, Mac OSX x86, HP-UX PA-RISC and NetBSD sparc64
Hooks up to APC opcode cache as a serialization handler (APC 3.1.7+)
Compatible with PHP 5.2 – 5.6

Implementation details

Storing complex PHP data structures like arrays of associative arrays with the standard PHP serializer is not very space efficient. The main reasons in order of significance are (at least in our applications):

Array keys are repeated redundantly.
Numerical values are plain text.
Human readability adds some overhead.

Igbinary uses two specific strategies to minimize the size of the serialized output.

Repetitive strings are stored only once. Collections of objects benefit significantly from this. See "compact_strings" option.
Numerical values are stored in the smallest primitive data type available: 123 = int8_t, 1234 = int16_t, 123456 = int32_t ... and so on.
( Well, it is not human readable ;)

How to use

Add the following lines to your php.ini:

; Load igbinary extension
extension=igbinary.so

; Use igbinary as session serializer
session.serialize_handler=igbinary

; Enable or disable compacting of duplicate strings
; The default is On.
igbinary.compact_strings=On

; Use igbinary as serializer in APC cache (3.1.7 or later)
;apc.serializer=igbinary

.. and in your php code replace serialize and unserialize function calls with igbinary_serialize and igbinary_unserialize.

Installing

Note: Sometimes phpize must be substituted with phpize5. In such cases the following option must be given to configure script: "--with-php-config=.../php-config5"

phpize
`./configure:
- With GCC: ./configure CFLAGS="-O2 -g" --enable-igbinary
- With ICC (Intel C Compiler) ./configure CFLAGS=" -no-prec-div -O3 -xO -unroll2 -g" CC=icc --enable-igbinary
- With clang: ./configure CC=clang CFLAGS="-O0 -g" --enable-igbinary
make
make test
make install
igbinary.so is installed to the default extension directory

To run APCu test

# go to modules directory
cd modules

# ... and create symlink to apcu extension
# it will be loaded during test suite
/opt/lib/php/extensions/no-debug-non-zts-20121212/apcu.so

Similar approach should work for APC.

Bugs & Contributions

Mailing list for bug reports and other development discussion can be found at http://groups.google.com/group/igbinary

Fill bug reports at https://github.com/igbinary/igbinary/issues

The preferred ways for contributions are pull requests and email patches (in git format). Feel free to fork at http://github.com/igbinary/igbinary

Utilizing in other extensions

Igbinary can be called from other extensions fairly easily. Igbinary installs its header file to ext/igbinary/igbinary.h. There are just two straighforward functions: igbinary_serialize and igbinary_unserialize. Look at igbinary.h for prototypes and usage.

Add PHP_ADD_EXTENSION_DEP(yourextension, igbinary) to your config.m4 in case someone wants to compile both of them statically into php.

Trivia

Where does the name "igbinary" come from? There was once a similar project called fbinary but it has disappeared from the Internet a long time ago. Its architecture wasn't particularly clean either. IG is an abbreviation for a finnish social networking site IRC-Galleria (http://irc-galleria.net/)

igbinary's People

Contributors

Stargazers

Watchers

igbinary's Issues

ERROR: Cannot run tests without CLI sapi.

I have here some questions...

here must be CLI enable or disable in php.ini or igbinary.ini??

thanks

Segmentation fault

I compile php with igbinary master code, use follow code, echo "Segmentation fault". :-(

plz fix it ASAP. :-)

array ( 0 => 2450, 1 => 2184, 2 => 2227, 3 => 2228, 4 => 2293, 5 => 2369, 6 => 2421, 7 => 2426, 8 => 2447, 9 => 2478, 10 => 2477, 11 => 2486, 12 => 2489, 13 => 2490, 14 => 2491, 15 => 2494, 16 => 2493, 17 => 2492, 18 => 2422, 19 => 2446, 20 => 2431, 21 => 2275, 22 => 2274, 23 => 2235, 24 => 2518, 25 => 2520, 26 => 2528, 27 => 2533, 28 => 2544, 29 => 2545, 30 => 2546, 31 => 2596, 32 => 2597, 33 => 2598, 34 => 2630, 35 => 2652, 36 => 2676, 37 => 2682, 38 => 2292, 39 => 2683, 40 => 2648, 41 => 2700, 42 => 2606, 43 => 2706, 44 => 2713, 45 => 2714, 46 => 2736, 47 => 2229, 48 => 2742, 49 => 2758, 50 => 2445, 51 => 2785, 52 => 2814, 53 => 2819, 54 => 2820, 55 => 2480, 56 => 2821, 57 => 2838, 58 => 2898, 59 => 2901, 60 => 2902, 61 => 2941, 62 => 2943, 63 => 2965, 64 => 2973, 65 => 2872, 66 => 3018, 67 => 3019, 68 => 3020, 69 => 3021, 70 => 3022, 71 => 3023, 72 => 3035, 73 => 3042, 74 => 3046, 75 => 3045, 76 => 3056, 77 => 3082, 78 => 3087, 79 => 3089, 80 => 2481, 81 => 3096, 82 => 3097, 83 => 3098, 84 => 3100, 85 => 3102, 86 => 3116, 87 => 3142, 88 => 2883, 89 => 3154, 90 => 3158, 91 => 3129, 92 => 3168, 93 => 3173, 94 => 3189, 95 => 3058, 96 => 3231, 97 => 3232, 98 => 3233, 99 => 3251, 100 => 3252, 101 => 3253, 102 => 3276, 103 => 3277, 104 => 3338, 105 => 3346, 106 => 3241, 107 => 3364, 108 => 3366, 109 => 3375, 110 => 3378, 111 => 3377, 112 => 3386, 113 => 3394, 114 => 3405, 115 => 3415, 116 => 3417, 117 => 3418, 118 => 3424, 119 => 3434, 120 => 3435, 121 => 3439, 122 => 3442, 123 => 3230, 124 => 3379, 125 => 3421, 126 => 3370, 127 => 3229, 128 => 3412, 129 => 3325, 130 => 3227, 131 => 3451, 132 => 3452, 133 => 2882, 134 => 2942, 135 => 3454, 136 => 2684, 137 => 3101, 138 => 2566, 139 => 3458, 140 => 3459, 141 => 2625, 142 => 2850, 143 => 3373, 144 => 2482, 145 => 2476, 146 => 2371, 147 => 2904, 148 => 3446, 149 => 3527, 150 => 3294, 151 => 3191, 152 => 3333, 153 => 2989, 154 => 3078, 155 => 3390, 156 => 2905, 157 => 3099, 158 => 2823, 159 => 2826, 160 => 2896, 161 => 2936, 162 => 2934, 163 => 2162, 164 => 2906, 165 => 2294, 166 => 2449, 167 => 1940, 168 => 2799, 169 => 2394, 170 => 1979, 171 => 3469, 172 => 3471, 173 => 3447, 174 => 3363, 175 => 3404, 176 => 2903, 177 => 3477, 178 => 3478, 179 => 2813, 180 => 3485, 181 => 2935, 182 => 3403, 183 => 3497, 184 => 3501, 185 => 3504, 186 => 3365, 187 => 3511, 188 => 3512, 189 => 3510, 190 => 3286, 191 => 3516, 192 => 3517, 193 => 3519, 194 => 3522, 195 => 3523, 196 => 2940, 197 => 3413, 198 => 2931, 199 => 2759, 200 => 3529, 201 => 3591, 202 => 3608, 203 => 3607, 204 => 3616, 205 => 3619, 206 => 3620, 207 => 3621, 208 => 3623, 209 => 3624, 210 => 3631, 211 => 3414, 212 => 3376, 213 => 3685, 214 => 3433, 215 => 3432, 216 => 3334, 217 => 3419, 218 => 3702, 219 => 3509, 220 => 2451, 221 => 3147, 222 => 3228, 223 => 3717, 224 => 3725, 225 => 3731, 226 => 3748, 227 => 3741, 228 => 3763, 229 => 3754, 230 => 3834, 231 => 3788, 232 => 3789, 233 => 3793, 234 => 3794, 235 => 3799, 236 => 3802, 237 => 3806, 238 => 3808, 239 => 3810, 240 => 3811, 241 => 3923, 242 => 3924, 243 => 3820, 244 => 3823, 245 => 3824, 246 => 3825, 247 => 3826, 248 => 3795, 249 => 3832, 250 => 3833, 251 => 3835, 252 => 3838, 253 => 3861, 254 => 3867, 255 => 3868, 256 => 3871, 257 => 3874, 258 => 3875, 259 => 3883, 260 => 3884, 261 => 3885, 262 => 3886, 263 => 3887, 264 => 3888, 265 => 3894, 266 => 3893, 267 => 3895, 268 => 3896, 269 => 3897, 270 => 3898, 271 => 3899, 272 => 3900, 273 => 3901, 274 => 3902, 275 => 3922, 276 => 3906, 277 => 3908, 278 => 3911, 279 => 3916, 280 => 3917, 281 => 3918, 282 => 3925, 283 => 3926, 284 => 3927, 285 => 3928, 286 => 3932, 287 => 3933, 288 => 3934, 289 => 3935, 290 => 3936, 291 => 3938, 292 => 3948, 293 => 3949, 294 => 3950, 295 => 3951, 296 => 3952, 297 => 3953, 298 => 3954, 299 => 3955, 300 => 3957, 301 => 3958, 302 => 3960, 303 => 3975, 304 => 3981, 305 => 3987, 306 => 3988, 307 => 3989, 308 => 3993, 309 => 3994, 310 => 3996, 311 => 3997, 312 => 4000, 313 => 4001, 314 => 4002, 315 => 4004, 316 => 4009, 317 => 4010, 318 => 4016, 319 => 4024, 320 => 4026, 321 => 4027, 322 => 4028, 323 => 4030, 324 => 4031, 325 => 4032, 326 => 4035, 327 => 4036, 328 => 4041, 329 => 4043, 330 => 4046, 331 => 4048, 332 => 4049, 333 => 4050, 334 => 4051, 335 => 4052, 336 => 4053, 337 => 4054, 338 => 4055, 339 => 4068, 340 => 4069, 341 => 4070, 342 => 4071, 343 => 4072, 344 => 4073, 345 => 4074, 346 => 4075, 347 => 4076, 348 => 4077, 349 => 4078, 350 => 4079, 351 => 4080, 352 => 4081, 353 => 4082, 354 => 4083, 355 => 4084, 356 => 4085, 357 => 4086, 358 => 4096, 359 => 4098, 360 => 4099, 361 => 4126, 362 => 4131, 363 => 4132, 364 => 4137, 365 => 4185, 366 => 4241, 367 => 4242, 368 => 4251, 369 => 4258, 370 => 4259, 371 => 4260, 372 => 4261, 373 => 4559, 374 => 4563, 375 => 4571, 376 => 4574, 377 => 4575, 378 => 4576, 379 => 4589, 380 => 4590, 381 => 4591, 382 => 4593, 383 => 4594, 384 => 4600, 385 => 4602, 386 => 4608, 387 => 4610, 388 => 4625, 389 => 4630, 390 => 4631, ), ); $mc = new Memcached; $mc->addServer('192.168.0.201', 7380); $mc->setMulti($datas, 3600); ?>

Php.ini for Memcached:

memcached

memcached support => enabled
Version => 2.0.0-dev
libmemcached version => 0.38
Session support => no
igbinary support => yes
json support => no

Directive => Local Value => Master Value
memcached.compression_factor => 1.3 => 1.3
memcached.compression_threshold => 2048 => 2048
memcached.compression_type => fastlz => fastlz
memcached.serializer => igbinary => igbinary

Failed Tests (Solaris/SPARC)

=====================================================================
PHP         : /tools/webapps/php/5.6.4_fpm/bin/php
PHP_SAPI    : cli
PHP_VERSION : 5.6.4
ZEND_VERSION: 2.6.0
PHP_OS      : SunOS - SunOS naicwebu 5.10 Generic_150400-17 sun4v
INI actual  : /tools/webapps/Source/php-igbinary/igbinary-1.0.2/tmp-php.ini
More .INIs  :
CWD         : /tools/webapps/Source/php-igbinary/igbinary-1.0.2
Extra dirs  :
VALGRIND    : Not used
=====================================================================
TIME START 2015-01-15 10:11:16
=====================================================================
PASS Check for igbinary presence [tests/001.phpt]
PASS Check for null serialisation [tests/002.phpt]
PASS Check for bool serialisation [tests/003.phpt]
PASS Check for integer serialisation [tests/004.phpt]
PASS Check for double serialisation [tests/005.phpt]
PASS Check for simple string serialization [tests/006.phpt]
PASS Check for simple array serialization [tests/007.phpt]
PASS Check for array+string serialization [tests/008.phpt]
FAIL Check for reference serialisation [tests/009.phpt]
PASS Array test [tests/010.phpt]
PASS Object test [tests/012.phpt]
PASS Object-Array test [tests/013.phpt]
PASS Object-Reference test [tests/014.phpt]
PASS Check for serialization handler [tests/015.phpt]
PASS Check for serialization handler, ini-directive [tests/015b.phpt]
PASS Object test, __sleep [tests/016.phpt]
PASS Object test, __wakeup [tests/017.phpt]
PASS Object test, __sleep error cases [tests/018.phpt]
PASS Object test, __autoload [tests/019.phpt]
PASS Object test, incomplete class [tests/020.phpt]
PASS Object Serializable interface [tests/021.phpt]
PASS Object test, unserialize_callback_func [tests/022.phpt]
PASS Resource [tests/023.phpt]
FAIL Recursive objects [tests/024.phpt]
FAIL Object test, array of objects with __sleep [tests/025.phpt]
FAIL Cyclic array test [tests/026.phpt]
PASS Check for serialization handler [tests/027.phpt]
PASS Serialize object into session, full set [tests/028.phpt]
PASS Igbinary module info [tests/029.phpt]
PASS Unserialize invalid data [tests/030.phpt]
PASS Object Serializable interface throws exceptions [tests/031.phpt]
FAIL Object test, __sleep and __wakeup exceptions [tests/032.phpt]
PASS Object test, cyclic references [tests/033.phpt]
PASS Unserialize invalid random data [tests/034.phpt]
SKIP Profiling perf test. [tests/035.phpt] reason: set TEST_PERFORMANCE=1 environment to enable trivial performance test
PASS b0rked random data test [tests/040.phpt]
=====================================================================
TIME END 2015-01-15 10:11:24

=====================================================================
TEST RESULT SUMMARY
---------------------------------------------------------------------
Exts skipped    :    0
Exts tested     :   36
---------------------------------------------------------------------

Number of tests :   36                35
Tests skipped   :    1 (  2.8%) --------
Tests warned    :    0 (  0.0%) (  0.0%)
Tests failed    :    5 ( 13.9%) ( 14.3%)
Expected fail   :    0 (  0.0%) (  0.0%)
Tests passed    :   30 ( 83.3%) ( 85.7%)
---------------------------------------------------------------------
Time taken      :    8 seconds
=====================================================================

=====================================================================
FAILED TEST SUMMARY
---------------------------------------------------------------------
Check for reference serialisation [tests/009.phpt]
Recursive objects [tests/024.phpt]
Object test, array of objects with __sleep [tests/025.phpt]
Cyclic array test [tests/026.phpt]
Object test, __sleep and __wakeup exceptions [tests/032.phpt]
=====================================================================

I'm guessing the failures have to do with misalignment on SPARC architecture based on the warning that GCC throws out:

In file included from /tools/webapps/php/5.6.4_fpm/include/php/Zend/zend_API.h:30,
                 from /tools/webapps/php/5.6.4_fpm/include/php/main/php.h:39,
                 from /tools/webapps/Source/php-igbinary/igbinary-1.0.2/igbinary.c:11:
/tools/webapps/php/5.6.4_fpm/include/php/Zend/zend_execute.h: In function `zend_vm_stack_new_page':
/tools/webapps/php/5.6.4_fpm/include/php/Zend/zend_execute.h:189: warning: cast increases required alignment of target type
/tools/webapps/php/5.6.4_fpm/include/php/Zend/zend_execute.h: In function `zend_vm_stack_frame_base':
/tools/webapps/php/5.6.4_fpm/include/php/Zend/zend_execute.h:267: warning: cast increases required alignment of target type
/tools/webapps/php/5.6.4_fpm/include/php/Zend/zend_execute.h: In function `zend_vm_stack_free_int':
/tools/webapps/php/5.6.4_fpm/include/php/Zend/zend_execute.h:272: warning: cast increases required alignment of target type
/tools/webapps/php/5.6.4_fpm/include/php/Zend/zend_execute.h: In function `zend_vm_stack_free':
/tools/webapps/php/5.6.4_fpm/include/php/Zend/zend_execute.h:284: warning: cast increases required alignment of target type

SKIP APC serializer registration [tests/igbinary_045.phpt] reason: APC not loaded

Hello,

Just ran a test. I have APC.so extension installed. But I got this message when I did "make test":
SKIP APC serializer registration [tests/igbinary_045.phpt] reason: APC not loaded

Something wrong?

php 5.3 support

Any chance that igbinary will work with php 5.3 in the near future?
I get warnings:
igbinary_unserialize_header: version mismatch: 35064129 vs 2

Failed tests for PHP 5.4.0RC7-dev

Two tests fail for PHP 5.4.0RC7-dev.

FAIL Recursive objects [ext/igbinary/tests/024.phpt]: No ouput instead of expected.

FAIL Unserialize backwards compatible with v1. [ext/igbinary/tests/unserialize_v1_compatible.phpt]:

Differing unserialized: object
Expected:
object(stdClass)#0 (4) {
  ["0"]=>
  int(1)
  ["1"]=>
  int(2)
  ["2"]=>
  int(3)
  ["3"]=>
  int(4)
}

Actual:
object(stdClass)#0 (4) {
  [0]=>
  int(1)
  [1]=>
  int(2)
  [2]=>
  int(3)
  [3]=>
  int(4)
}

Failed test in make test (ubuntu 9.04, zend server ce, php 5.3.3)

TEST RESULT SUMMARY

Exts skipped : 0

Exts tested : 22

Number of tests : 38 37
Tests skipped : 1 ( 2.6%) --------
Tests warned : 0 ( 0.0%) ( 0.0%)
Tests failed : 2 ( 5.3%) ( 5.4%)
Expected fail : 0 ( 0.0%) ( 0.0%)

Tests passed : 35 ( 92.1%) ( 94.6%)

Time taken : 0 seconds

FAILED TEST SUMMARY

Check for reference serialisation [tests/009.phpt]
Cyclic array test [tests/026.phpt]