Comments (10)
The error is coming from here:
usearch/python/usearch/__init__.py
Lines 103 to 110 in 708fede
Here's where that download URL is constructed:
usearch/python/usearch/__init__.py
Lines 25 to 51 in 708fede
from usearch.
That worked perfectly!
$ pip install usearch
Collecting usearch
Downloading usearch-2.10.3-cp311-cp311-macosx_11_0_arm64.whl (400 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 400.3/400.3 kB 2.8 MB/s eta 0:00:00
Requirement already satisfied: numpy in /Users/simon/.local/share/virtualenvs/foop-q39PfZK-/lib/python3.11/site-packages (from usearch) (1.24.3)
Requirement already satisfied: tqdm in /Users/simon/.local/share/virtualenvs/foop-q39PfZK-/lib/python3.11/site-packages (from usearch) (4.65.0)
Installing collected packages: usearch
Successfully installed usearch-2.10.3
$ python
Python 3.11.8 (main, Feb 6 2024, 21:21:21) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import usearch
>>> usearch.sqlite_path()
/Users/simon/.local/share/virtualenvs/foop-q39PfZK-/lib/python3.11/site-packages/usearch/__init__.py:93: UserWarning: Will download `usearch_sqlite` binary from GitHub.
warnings.warn("Will download `usearch_sqlite` binary from GitHub.", UserWarning)
'/Users/simon/.local/share/virtualenvs/foop-q39PfZK-/bin/usearch_binaries/usearch_sqlite'
>>> import sqlite3
>>> conn = sqlite3.connect(":memory:")
>>> conn.enable_load_extension(True)
>>> conn.load_extension(usearch.sqlite_path())
>>> [fn[0] for fn in conn.execute('pragma function_list').fetchall()]
['pow', 'group_concat', 'group_concat', 'json_type', 'json_type', 'jsonb_set', 'julianday', 'ntile', 'nullif', 'sqlite_compileoption_get', 'json_valid', 'json_valid', 'json_quote', 'json_patch', '->', 'json_array', 'current_timestamp', 'power', 'sqlite_compileoption_used', 'json_remove', 'jsonb_patch', 'json_object', 'json_insert', '->>', 'jsonb_array', 'sin', 'sum', 'quote', 'printf', 'likelihood', 'json_replace', 'jsonb_remove', 'jsonb_object', 'jsonb_insert', 'json_extract', 'last_value', 'rank', 'sign', 'sqrt', 'sinh', 'tan', 'round', 'round', 'rtrim', 'rtrim', 'jsonb_replace', 'jsonb_extract', 'nth_value', 'tanh', 'random', 'trim', 'trim', 'time', 'radians', 'trunc', 'total', 'substr', 'substr', 'replace', 'unhex', 'unhex', 'upper', 'subtype', 'typeof', 'load_extension', 'load_extension', 'json_group_array', 'avg', 'abs', 'octet_length', 'json_group_object', 'jsonb_group_array', 'json_array_length', 'json_array_length', 'strftime', 'atan', 'asin', 'acos', 'substring', 'substring', 'randomblob', 'unicode', 'jsonb_group_object', 'timediff', 'percent_rank', 'row_number', 'atanh', 'asinh', 'acosh', 'cos', 'atan2', 'string_agg', 'last_insert_rowid', 'sqlite_log', 'unlikely', 'json_error_position', 'cosh', 'ceil', 'char', 'unixepoch', 'exp', 'count', 'count', 'date', 'concat', 'ceiling', 'total_changes', 'changes', 'sqlite_version', 'degrees', 'floor', 'coalesce', 'glob', 'zeroblob', 'hex', 'iif', 'sqlite_source_id', 'concat_ws', 'format', 'datetime', 'cume_dist', 'ln', 'instr', 'json', 'dense_rank', 'log', 'log', 'ifnull', 'jsonb', 'current_date', 'current_time', 'lag', 'lag', 'lag', 'mod', 'log2', 'like', 'like', 'max', 'max', 'min', 'min', 'lead', 'lead', 'lead', 'log10', 'lower', 'ltrim', 'ltrim', 'first_value', 'pi', 'length', 'likely', 'json_set', 'distance_inner_i8', 'distance_cosine_i8', 'distance_inner_f16', 'distance_sqeuclidean_f16', 'distance_divergence_f32', 'fts3_tokenizer', 'fts3_tokenizer', 'distance_divergence_i8', 'bm25', 'distance_jaccard_binary', 'distance_sqeuclidean_i8', 'highlight', 'rtreenode', 'rtreecheck', 'geopoly_regular', 'matchinfo', 'matchinfo', 'geopoly_debug', 'geopoly_bbox', 'distance_hamming_unicode', 'fts5_source_id', 'geopoly_contains_point', 'geopoly_svg', 'distance_haversine_meters', 'snippet', 'geopoly_overlap', 'distance_sqeuclidean_f64', 'distance_hamming_bytes', 'distance_levenshtein_unicode', 'distance_cosine_f32', 'distance_cosine_f16', 'rtreedepth', 'geopoly_area', 'match', 'geopoly_blob', 'optimize', 'geopoly_within', 'distance_inner_f64', 'geopoly_json', 'offsets', 'distance_cosine_f64', 'geopoly_xform', 'distance_divergence_f16', 'fts5', 'distance_sqeuclidean_f32', 'geopoly_ccw', 'distance_hamming_binary', 'distance_levenshtein_bytes', 'distance_divergence_f64', 'geopoly_group_bbox', 'distance_inner_f32']
>>> conn.execute('select distance_cosine_f32(?, ?)', (repr([1.1, 2, 3]), repr([3.1, 1, 2]))).fetchall()
[(0.20811247825622559,)]
from usearch.
I am trying to avoid a breaking change in how the SQLite module is loaded, and it really overcomplicates things. Working on a hot-fix now 🤗
from usearch.
@simonw there are a couple of issues here.
- SQLite will add
.dylib
suffix at the end of the name looking for a.dylib.dylib
file. - Even after succesfull download, my M2 Mac complains about the build compatibility.
>>> conn.load_extension(usearch.sqlite.removesuffix('.dylib'))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
sqlite3.OperationalError: dlopen(/var/folders/r0/mw8phgys4lg_60kgh8xm_x780000gn/T/usearch_sqlite_macos_arm64_2.10.0.dylib, 0x000A): tried: '/var/folders/r0/mw8phgys4lg_60kgh8xm_x780000gn/T/usearch_sqlite_macos_arm64_2.10.0.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/var/folders/r0/mw8phgys4lg_60kgh8xm_x780000gn/T/usearch_
That doesn't happen with local builds, so it's somewhere in the CI. Still checking 😉
from usearch.
🎉 This issue has been resolved in version 2.10.1 🎉
The release is available on GitHub release
Your semantic-release bot 📦🚀
from usearch.
I am gonna change the import method, which is technical a breaking change and should trigger a major release, but assuming the low adoption of the SQLite API, I hope it's fine 🤗
from usearch.
It's not quite working for me still:
```pycon
>>> import usearch
/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/site-packages/usearch/__init__.py:100: UserWarning: Will download `usearch_sqlite` binary from GitHub.
warnings.warn("Will download `usearch_sqlite` binary from GitHub.", UserWarning)
>>> import sqlite3
>>> conn = sqlite3.connect(":memory:")
>>> conn.enable_load_extension(True)
>>> conn.load_extension(usearch.sqlite_path())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'usearch' has no attribute 'sqlite_path'
>>> usearch.sqlite
'/var/folders/x6/31xf1vxj0nn9mxqq8z0mmcfw0000gn/T/usearch_sqlite_macos_arm64_2.10.0.dylib'
>>> conn.load_extension(usearch.sqlite)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
sqlite3.OperationalError: dlopen(/var/folders/x6/31xf1vxj0nn9mxqq8z0mmcfw0000gn/T/usearch_sqlite_macos_arm64_2.10.0.dylib.dylib, 0x000A): tried: '/var/folders/x6/31xf1vxj0nn9mxqq8z0mmcfw0000gn/T/usearch_sqlite_macos_arm64_2.10.0.dylib.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/var/folders/x6/31xf1vxj0nn9mxqq8z0mmcfw0000gn/T/usearch_sqlite_macos_arm64_2.10.0.dylib.dylib' (no such file), '/var
>>> conn.lead_extension(usearch.sqlite.replace('.dylib', ''))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'sqlite3.Connection' object has no attribute 'lead_extension'. Did you mean: 'load_extension'?
>>> conn.load_extension(usearch.sqlite.replace('.dylib', ''))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
sqlite3.OperationalError: dlopen(/var/folders/x6/31xf1vxj0nn9mxqq8z0mmcfw0000gn/T/usearch_sqlite_macos_arm64_2.10.0.dylib, 0x000A): tried: '/var/folders/x6/31xf1vxj0nn9mxqq8z0mmcfw0000gn/T/usearch_sqlite_macos_arm64_2.10.0.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/var/folders/x6/31xf1vxj0nn9mxqq8z0mmcfw0000gn/T/usearch_
It says it got x86_64
but needed arm64
.
Also note I had to strip the .dylib
from the end of the path - plus usearch.sqlite_path()
has been renamed usearch.sqlite` - that's a bug in this README:
Lines 28 to 33 in 3fd5184
from usearch.
I tried downloading this: https://github.com/unum-cloud/usearch/releases/download/v2.10.1/usearch_sqlite_macos_arm64_2.10.1.dylib
Then opening it like this:
>>> conn.load_extension("/tmp/usearch_sqlite_macos_arm64_2.10.1")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
sqlite3.OperationalError: dlopen(/tmp/usearch_sqlite_macos_arm64_2.10.1.dylib, 0x000A): tried: '/tmp/usearch_sqlite_macos_arm64_2.10.1.dylib' (code signature in <DCF25A47-045B-3D73-BE3E-6EF40C87A897> '/private/tmp/usearch_sqlite_macos_arm64_2.10.1.dylib' not valid for use in process: library load disallowed by system policy), '/System/Volumes/Preboot/Cryptexes
I fixed that error by opening up the macOS System Preferences and going to Security and allowing that binary. On a second attempt:
>>> conn.load_extension("/tmp/usearch_sqlite_macos_arm64_2.10.1")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
sqlite3.OperationalError: dlsym(0x81325550, sqlite3_usearchsqlitemacosarm_init): symbol not found
from usearch.
Yes, the right architecture wasn't selected in CI for the 2.10.0 build, that you are using. But the 2.10.1 should be fixing that. The CI can take a couple of hours to complete all the tests and I will try again afterwards.
@simonw, your log, however, is quite handy, as it highlights another issue - SQLite is searching for a weird symbol name when loading the library - sqlite3_usearchsqlitemacosarm_init
. That will be an issue even with the new installation. If you git clone
and then pip install -e .
locally, the issue won't happen. Alternatively, you can rename the binary to usearch_sqlite.dylib
locally, it may pass.
Give me a few hors to polish this. Sorry for inconvenience, debugging GitHub CI is an absolute nightmare 🧠 🔫
from usearch.
@simonw, can you please check the 2.10.3? It should work fine.
from usearch.
Related Issues (20)
- Bug: Non of methods are working in Multi-threaded envirionment HOT 12
- Bug: Index restore view=True does not work for shared user index HOT 3
- Usearch in Rust fails to compile for Android
- Bug: JAVA Search not match after deleting key HOT 3
- Bug: 'Illegal Instruction' while restoring an index from disk HOT 1
- Bug: Missing results after save + view + filtered_search HOT 5
- Bug: Index build is 40% slower when SIMSIMD is enabled on Mac M1 Pro HOT 10
- Bug: error loading SQLite extension without python
- Bug: [Rust] all distances are zero HOT 2
- Feature: Add Python documentation regarding how to search with custom metrics with more than two variables
- Bug: `nodes_capacity_` is not set in header when saving to stream HOT 1
- Bug: Rust bindings: view_from_buffer should be marked as unsafe. HOT 1
- Bug: `PersistAndRestore` C# test fails on MacOS HOT 7
- Bug: Compilation errors for nodejs on v2.13.0 HOT 1
- Bug: Rust wrapper fails to load index correctly from buffer or file. HOT 4
- Add documentation for eval module
- GPU Acceleration
- Bug: pip install can't find correct version , usearch can't support ubuntu x86_64? HOT 1
- Feature: support all data types listed in scalar_kind_t in index_dense HOT 1
- Feature: Add swift/obj-c bindings for filteredSearch HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from usearch.