iden3 / ffjavascript Goto Github PK

Hi! I am opening this issue to see if you'd be happy to receive contribution for some benchmarks. The zk-harness people already developed something https://github.com/zkCollective/zk-Harness/blob/feat/zkalc/circom/scripts/arithmetics.js#L28 but their quality is subpar. I'd be interesting in adding an "npm bench" that can be run to provide benchmarks for finite field arithmetic.

Incidentally this may also allow for comparison with other projects and uncovering significant optimisations! :)

I'm creating a react project which is using this library as a dependency, I believe the fix would be to change "process.browser " to typeof window === 'undefined' , as process.browser has been deprecated

snarkJS: Error: Curve not supported: 4002409555221667610661788685990436838824310417315709160196267947217350388461752218928110433526561144555515641069568
at getCurveFromQ (/home/root/snarkjs/build/cli.cjs:765:15)
at readPTauHeader (/home/root/snarkjs/build/cli.cjs:842:25)
at async contribute (/home/root/snarkjs/build/cli.cjs:2408:43)
at async Object.powersOfTauContribute [as action] (/home/root/snarkjs/build/cli.cjs:8619:12)
at async clProcessor (/home/root/snarkjs/build/cli.cjs:304:27)

Are there typescript bindings available?

First, thanks for this great library that takes a lot of work off my hands :). One issue with it though is that it currently does not work with many Babel configurations due to the way Babel hoists polyfills to the top of the bundle (exact details and extended discussion on why this shouldn't be changed inside of Babel here ).

The fix to make ffjavascript work with any Babel config would be quite simple and is something we already implemented in a fork of our own here, it just entails replacing async-await with a Promise within the worker threads. This doesn't require a lot of effort and shouldn't break anything for anyone, as any environment that supports async-await also supports Promise. Happy to open a PR for this myself unless there's some reason why ffjavascript prefers not to implement this. Thanks! 😄

It seems that the line of code at the bottom can be moved to the beginning of the buildBn128 function so that after the curve is cached, the code before it is not executed each time.

Moving that line to the beginning could potentially make proof verification and generation much faster (after the curve is cached).

The same happens within buildBls12381.

This is a real problem when writing code that uses the libraries on top of this one. For instance I want to generate a babyJubjub key and serialize it, then deserialize it and use it in some circuit. Right now I am struggling to figure out what function will do the job.

I tried calling millerLoop during the verify process, but the result is all 0：

async function groth16Verify$1(_vk_verifier, _publicSignals, _proof, logger) {
   
    ....
    const vk_alpha_1 = curve.G1.fromObject(vk_verifier.vk_alpha_1);
    const vk_beta_2 = curve.G2.fromObject(vk_verifier.vk_beta_2);

    const alphaBeta_tmp = await curve.millerLoop(vk_alpha_1, vk_beta_2);

    console.log("alphaBeta", curve.Gt.toObject(alphaBeta_tmp));

    const res = await curve.pairingEq(
        curve.G1.neg(pi_a) , pi_b,
        cpub , vk_gamma_2,
        pi_c , vk_delta_2,

        vk_alpha_1, vk_beta_2
    );
    ...

got result blow:

alphaBeta [
  [ [ 0n, 0n ], [ 0n, 0n ], [ 0n, 0n ] ],
  [ [ 0n, 0n ], [ 0n, 0n ], [ 0n, 0n ] ]
]

Hi there! We were using ffjavascript as a dependency in our p0tion codebase and we recently run into a weird runtime problem:

nico@instance-5:~$ phase2cli --version
(node:31877) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
/home/sachin/.nvm/versions/node/v18.19.0/lib/node_modules/@p0tion/phase2cli/node_modules/web-worker/node.js:17
import URL from 'url';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at internalCompileFunction (node:internal/vm:73:18)
    at wrapSafe (node:internal/modules/cjs/loader:1274:20)
    at Module._compile (node:internal/modules/cjs/loader:1320:27)
    at Module._extensions..js (node:internal/modules/cjs/loader:1414:10)
    at Module.load (node:internal/modules/cjs/loader:1197:32)
    at Module._load (node:internal/modules/cjs/loader:1013:12)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:202:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:195:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:336:24)
    at async loadESM (node:internal/process/esm_loader:34:7)

We noticed that when installing p0tion, ffjavascript would install the latest web-worker version (1.3.0) because the package.json has "web-worker": "^1.2.0" and that would cause the SyntaxError: Cannot use import statement outside a module error.

In our end we decided to rewrite a couple of ffjavascript and uninstall the package but I feel this problem might appear in other codebases as well. That is why I decided to report it here.

Please let me know if I can help out in anything.

Best,

Nico

I haven't tried and tested yet tho, for instant reporting.

Hi! The library returns a seemingly wrong value for the multiplication in bn128 Fr.

Here's an example: (I assumed that it's using little endian)
It gives 21005330981446606953608831075862683397340720297895507971069995954406343750754 while I'm expecting 19138581828712882593908706925461583697567205063300498062599267069621213248340.

Both the online calculator and the circom's * operation give the expected result.

const ffjavascript = require("ffjavascript");

async function checkBn128() {
        const a =
                5299619240641551281634865583518297030282874472190772894086521144482721001553n;
        const b =
                16950150798460657717958625567821834550301663161624707787222815936182638968203n;

        const bn128 = await ffjavascript.getCurveFromName("bn128", true);
        const cBuf = bn128.Fr.mul(
                newBufferFromBigUInt256LE(a),
                newBufferFromBigUInt256LE(b)
        );
        const c = readBigUInt256LE(cBuf);
        // WRONG RESULT FOR c
        // Expecting 19138581828712882593908706925461583697567205063300498062599267069621213248340
        // reference: https://planetcalc.com/8326/
        //   circomlib BabyAdd also gives the same result.
        console.log("c:");
        console.log(c);
}

checkBn128();

/* util functions */
function readBigUInt256LE(buffer) {
        var value = 0n;
        for (var i = 0; i < 32; i++) {
                value += BigInt(buffer[i]) << BigInt(i * 8);
        }
        return value;
};

function newBufferFromBigUInt256LE(value) {
        var buffer = Buffer.alloc(32);
        for (var i = 0; i < 32; i++) {
                buffer[i] = Number(value & 0xffn);
                value = value >> 8n;
        }
        return buffer;
};

/* util function tests */
function testNewBufferFromBigUInt256LE() {
        const a =
                5299619240641551281634865583518297030282874472190772894086521144482721001553n;
        const aBuf = newBufferFromBigUInt256LE(a);
        console.log('a buffer');
        console.log(aBuf);
        // expecting 51, 70, 95, BB, F6, F3, 93, 28, B6, E0, 34, 05, 01, D8, B8, 2A, C1, 77, 62, 9D, E0, B2, AC, 4E, 9B, 73, 3E, D6, 6A, 7A, B7, 0B
        // reference: https://www.rapidtables.com/convert/number/decimal-to-hex.html
        return aBuf;
}

function testReadBigUInt256LE() {
        const aPrime = readBigUInt256LE(testNewBufferFromBigUInt256LE());
        console.log("aPrime:");
        console.log(aPrime);
}

// testNewBufferFromBigUInt256LE();
// testReadBigUInt256LE();

As you can see in this issue process.browser is deprecated, and you should use typeof window === "undefined".
vercel/next.js#5354 (comment)

When running react-scripts build in a React app the following error will occur in @iden3/binfileutils as well as circom_runtime:

node_modules/@iden3/binfileutils/node_modules/ffjavascript/src/engine_fft.js: Couldn't find a Program

Description of the Problem

Using leInt2Buff returns inaccurate results depending on the length of the input. I discovered this when constructing a buffer from a small value, eg BigInt(42). When using leInt2Buff(BigInt(42)), the result is Uint8Array(1) [1], whereas the expected result is Uint8Array(1) [42].

My solution in this case is to strictly specify the length passed into leInt2Buff (and to make sure its a multiple of 4). That being said, this issue might go undetected by a consumer of this library, and that could potentially impact the security of a zero knowledge application.

For example, if someone were to write a zk app that uses the pedersenHash of a value that is less than 4 bytes long (up to 2^32). If the primage depends on a Buffer constructed from this function there will be unintentional collisions, because for example leInt2Buff(BigInt(42)) == leInt2Buff(BigInt(216)) == Uint8Array(1) [1] (and there are more repeated patterns up until you exceed 4 bytes, then the pattern changes again).

Steps to Reproduce

Run this script.

const { leInt2Buff } = require('ffjavascript').utils;

const num = BigInt(42);
console.log('num: ', num);
console.log('leInt2Buff len = undefined: ', leInt2Buff(num));
for (let i = 1; i < 33; i++) {
    console.log(`leInt2Buff len = ${i}: `, leInt2Buff(num, i));
}

The truncated results of the above script:

num:  42n
leInt2Buff len = undefined:  Uint8Array(1) [ 1 ]
leInt2Buff len = 1:  Uint8Array(1) [ 1 ]
leInt2Buff len = 2:  Uint8Array(2) [ 0, 1 ]
leInt2Buff len = 3:  Uint8Array(3) [ 0, 1, 1 ]
leInt2Buff len = 4:  Uint8Array(4) [ 42, 0, 0, 0 ]
leInt2Buff len = 5:  Uint8Array(5) [ 42, 0, 0, 0, 1 ]
leInt2Buff len = 6:  Uint8Array(6) [ 42, 0, 0, 0, 0, 1 ]
leInt2Buff len = 7:  Uint8Array(7) [
  42, 0, 0, 0,
   0, 1, 1
]
leInt2Buff len = 8:  Uint8Array(8) [
  42, 0, 0, 0,
   0, 0, 0, 0
]

The pattern of ones at the end of the array continues so on and so forth up until len = 32.

Here's another script, this time comparing 216 vs 42:

const num1 = BigInt(216);
console.log('num1: ', num1);
console.log('leInt2Buff len = undefined: ', leInt2Buff(num1));
console.log('leInt2Buff len = 32: ', leInt2Buff(num1, 32));

const num2 = BigInt(42);
console.log('num2: ', num2);
console.log('leInt2Buff len = undefined: ', leInt2Buff(num2));
console.log('leInt2Buff len = 32: ', leInt2Buff(num2, 32));

num1:  216n
leInt2Buff len = undefined:  Uint8Array(1) [ 1 ]
leInt2Buff len = 32:  Uint8Array(32) [
  216, 0, 0, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0
]
num2:  42n
leInt2Buff len = undefined:  Uint8Array(1) [ 1 ]
leInt2Buff len = 32:  Uint8Array(32) [
  42, 0, 0, 0, 0, 0, 0, 0, 0,
   0, 0, 0, 0, 0, 0, 0, 0, 0,
   0, 0, 0, 0, 0, 0, 0, 0, 0,
   0, 0, 0, 0, 0
]

I ran this script in a local project with the following package.json:

{
  "dependencies": {
    "@ethersproject/bignumber": "^5.5.0",
    "@ethersproject/wallet": "^5.5.0",
    "@types/node": "^17.0.19",
    "circomlibjs": "^0.1.1",
    "ffjavascript": "^0.2.48",
    "typescript": "^4.5.5"
  }
}

Hi. I work with EF and ethereumjs teams on supply chain security.

Last 6 months i've been helping to move the ETH ecosystem to native bigints. Their time has come. They are not less secure when compared to 3rd party js modules with regards to timing attacks etc. We're a few weeks away from a major ethereumjs-util release, ethers is also using native bigints in their beta.

Since almost every ETH app depend on either ethereumjs-util, or ethers, this would mean the majority of them would be covered.

So, it would be great if ffjavascript did the switch, because many important libraries depend on it.

I have developed a new web worker for nodejs with a bit enhancement over the original author.
it supports creating web worker from blob urls. so you don't have to make any weird complicated base64 urls
URL and blobs are supported in nodejs too nowdays

import Worker from 'whatwg-worker'
const code = 'console.log(1)'
const blob = new Blob([code], { type: 'text/javascript' })
const url = URL.createObjectURL(blob)

worker = new Worker(url)

Works the same way in both browser and nodejs
ofc if you would like to see fewer dependencies i highly suggest you to give this nodejs/node#43583 a 👍

When using with [email protected], i got the following error. Manually changing MEM_SIZE from 25 to 1000 fixes the issue.

Uncaught (in promise) LinkError: WebAssembly.instantiate(): memory import 0 is smaller than initial 1000, got 25
    at buildThreadManager (main.cjs:4935)
    at async buildEngine (main.cjs:6276)
    at async Object.buildBn128 (main.cjs:6348)
    at async getCurveFromQ (main.cjs:33)
    at async readHeaderGroth16 (main.cjs:341)
    at async readHeader (main.cjs:314)
    at async groth16Prove (main.cjs:744)
    at async Object.groth16FullProve [as fullProve] (main.cjs:1144)

I am running on:

• MBP Big Sur
• Brave 1.29.76 Chromium: 93.0.4577.58 (Official Build) (x86_64)
• Node 14.17.6
• npm 6.14.15

In the shift operation, bit length is compared to partition, but this is not reasonable.
For example, let p = 17, a = 5 and b = 6. Then run the following code, which will shift the 5 six bits to the left, but we're still looking for a rightward shift.
Therefore, we should use p/2 as our criterion, as described in the circom documentation.

Existing implementation:

shr(a, b) {
        if (b.lt(this.bitLength)) {
            return a.shiftRight(b);
        } else {
            const nb = this.p.minus(b);
            if (nb.lt(this.bitLength)) {
                return this.shl(a, nb);
            } else {
                return bigInt.zero;
            }
        }
    }

I haven't tried and tested yet tho, for instant reporting.

I'm trying to run a small script to generate zeros but can't due to mixed types error:

const bigInt = require('big-integer');
const { leInt2Buff } = require('ffjavascript').utils;

const roots = [...Array(2).keys()].map(_ => {
    leInt2Buff(bigInt('0'), 31);
});

I get the error:

Uncaught TypeError: Cannot mix BigInt and other types, use explicit conversions
    at leInt2Buff (/Users/drewstone/code/webb/protocol-solidity/node_modules/ffjavascript/build/main.cjs:3342:40)

Is there a specific Node version I should use or any idea how to get around this?

Hello, we are developing browser extension and have ffjavascript and a dependency in our node modules.
We have chrome extension manifest v3 and it's not possible to spawn a web worker inside service worker so we had to fork this repo to fix it https://github.com/CryptKeeperZK/ffjavascript.
For now this solution works only for browser and fails for node.

Is it possible to add support for extensions? I can create a PR for this case but first I'd like to hear the feedback or maybe there is some other solution to make it works.