monoclex / stringdb Goto Github PK
View Code? Open in Web Editor NEWStringDB is a modular, key/value pair archival DB designed to consume *tiny* amounts of ram & produce *tiny* databases.
License: MIT License
StringDB is a modular, key/value pair archival DB designed to consume *tiny* amounts of ram & produce *tiny* databases.
License: MIT License
most time Exception on TryGet Method.
the fllow is the test Code
StringDBTest.TestFileNameLengthEffects0(); //Test Entry
public class StringDBTest {
static StringDBTest() {
try {
//db = new LiteDatabase("Filename=TestDB.ldb;");//初始化数据库connection=shared
db = new DatabaseBuilder()
.UseIODatabase(StringDBVersion.Latest, "StringDBTest.db", out var optimalTokenSource)
.WithBuffer(1000)
.WithTransform(StringTransformer.Default, StringTransformer.Default);
} catch (Exception ex) {
Console.WriteLine(ex.Message);
}
}
public static IDatabase<string, string> db;
public static List<CTS1> DataGen(ref int current, int count) {
List<CTS1> list = new(count);
for (int i = 0; i < count; i++) {
var val = (current + i);
var key = $"0d3deed1234567890abcd{(100000000 - (current + i)):D12}";
list.Add(new(key, val));
}
current = current + count;
return list;
}
static Random r = new Random();
public static async void SelectTest() {
while (!IsExit()) {
await Task.Delay(r.Next(1, 10));//随机触发查询时间
var sw = Stopwatch.StartNew();
var rs = r.Next(0, 10000000);
var key = $"0d3deed1234567890abcd{(100000000 - rs):D12}";
db.TryGet(key, out string strVal);
//var rlt = db.Get(key);
if (sw.Elapsed.Seconds > 1)
Console.WriteLine($"查询时间大于1秒-{sw.ElapsedMilliseconds}ms");
Interlocked.Increment(ref QueryCount);
}
}
public static bool IsExit() {
return Interlocked.Read(ref exit) == 1;
}
static long exit = 0;
public static long QueryCount;
public static void TestFileNameLengthEffects0() {
//using var db1 = new LiteDatabase("Filename=TestDB0.ldb;");
var sw = Stopwatch.StartNew();
var swOutput = Stopwatch.StartNew();
for (int t = 0; t < 50; t++)
Task.Factory.StartNew(SelectTest);
int totalIdx = 26860000;
for (int i = 0; i < 10000; i++) {
var list = DataGen(ref totalIdx, 20000); //每2w一个批次
try {
db.InsertRange(list.Select(cts=> new KeyValuePair<string,string>(cts.Code, cts.ToString())).ToArray());
} catch (Exception ex) {
Console.WriteLine(ex.Message);
}
if (swOutput.Elapsed.TotalSeconds > 8) {
Console.WriteLine($" 当前数量:{totalIdx} 平均插入速度:{(totalIdx / sw.Elapsed.TotalSeconds)}op/s , 查询次数{(QueryCount / sw.Elapsed.TotalSeconds)}/s");
swOutput.Restart();
}
}
exit = 1;
}
/// <summary>
/// CodeTaskSmall
/// </summary>
public class CTS1 {
public CTS1(string code, int taskId) {
Code = code;
TaskId = taskId;
}
[LiteDB.BsonId]
public string Code;
[LiteDB.BsonField("t")]
public int TaskId;
}
}
a lack of async disturbs me.
currently, the biggest blocker of this feature is that you can't do concurrent "seek & read" operations on a Stream, and for IODatabases this would be an issue.
it is a theoretical possibility to add some kind of DatabaseFactory which produces single threaded IDatabases, and an AsyncDatabase would use this factory to create a database on demand and then read from it and close it, so concurrent operations could be done by having multiple file handles open, though it would be nice if there were alternatives.
additionally, the compatability issues of synchronous code and async is a bit disturbing, it would be nice if code didn't have to be duplicated between synchronous and asynchronous.
All keys data needs to fit in RAM ?
deleting stuff in a database would be nice.
however, there are multiple ways to do this:
moreover, an API for deleting would be complicated. currently, the suggested route is to pass in multiple ILazyLoader[]s to Delete, but then you'd have to manage existing ILazyLoaders to ensure that the relative positions they store get updated as well.
perhaps it'd be better if you'd be forced to close the database, and do separate deletion operations via an entirely different class, forcing developers to adapt to a different, more appropriate style.
StringDB isn't designed to be used to constantly add/remove stuff, and it'd be a more appropriate workflow to be forced to close the database, then modify it, then re-open it.
a current workaround is to and add all the things you want in the new database that aren't the deleted items.
Maybe You will be interested in hosting API Explorer online:
www.robiniadocs.com/d/stringdb
Hello, I've read over the main page and the example and I don't see a "file" option. Is there any way to save / restore the database from disk?
Thanks
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.