intern's Introduction

Intern

What is this?

A way of assigning a unique integer ID to each unique string, without collisions
Two-way lookup: ID => string, string => ID
Each string is stored only once in memory
Optional inlining of unsigned integer strings
Very low fragmentation via a custom block allocator
Minimal overhead per string: currently ~40 bytes, which could be lower at the cost of additional fragmentation
Fast: intern many millions of strings per second
String repository optimization based on frequency analysis (improve locality)
Support for snapshots (restore to a previous state)

Installation

$ cmake -G 'Unix Makefiles' -Wno-dev [OPTIONS]
$ make install

Options are:

-DBUILD_STATIC=1: Build a static library (libintern.a) rather than a shared library (libintern.so or libintern.dylib)
-DMMAP_PAGES=1: Allocate pages with mmap(2) rather than malloc(3)
-DPAGE_SIZE=4096: Set the page size
-DINLINE_UNSIGNED=1: Inline unsigned integers between 0 and INT_MAX
-DCMAKE_BUILD_TYPE=Release: Do a release build / enable optimization

Usage

Build your project with -lintern and include <intern/strings.h>.

See strings.h and optimize.h for more details.

Extra

Bindings for Go

License

MIT

intern's People

Contributors

Stargazers

Watchers

intern's Issues

Consider generating a static "image"

Hello,
I would like to use your framework to speedup code of a command interpreter on an embedded system. The specificity is that the amount of RAM is quite small, and that the set of strings is fixed.

The procedure would be;

on a development computer, take a list of strings and generate "something" which could be inserted into a C program. Basically as: struct ZZZ my_frozen_strings={{.val1 = XXX, .val2 = YYY}, ...};
on the target platform, add this 'image' and incorporate only the code permitting to search one string in the set. No grow / pruning / snapshot

Where should I start with ?

Regards

Pascal

Consider adding mechanism for user-defined allocation/hashing

This is a nice library. Thanks for making it!

A number of other C libraries I've used include facilities for users optionally specifying their own allocation and hashing methods. This can be quite useful for testing and portability. Perhaps you could consider adding something like this?

Recommend Projects