Giter VIP home page Giter VIP logo

Comments (4)

ejizba avatar ejizba commented on June 27, 2024 1

Hi @Dev-InSoft I'm not familiar with @anzp/azure-function-multipart, but yes request.formData() is the recommended way to parse form data. Could you perhaps give more detailed repro steps with sample data? Any idea what characters are missing? For example is it from the end of the data or is it maybe a difference in whitespace?

The biggest problem we have with form data right now is we don't support streaming http requests. I don't know if this is your problem, but if you are sending the request with "transfer-encoding" set to "chunked", that could explain it. We hope to support streams soon (tracked by #97).

from azure-functions-nodejs-library.

ejizba avatar ejizba commented on June 27, 2024 1

So it turns out this default limit of "1048576" is set by the busboy npm package. Your original package "@anzp/azure-function-multipart" and "undici" are both using "busboy" as a dependency, but it seems like only the former supports passing the config on to busboy. My current recommendation would be to use "busboy" directly if you need to configure the limits. The other alternative is to file an issue on undici so that we can configure the limits, but the fetch spec for formData doesn't allow any parameters so I'm skeptical we'd get traction.

Here's some rough code that worked for me. It's just a combination of your code and the busboy sample in their readme:

import { app, HttpRequest, HttpResponseInit, InvocationContext } from "@azure/functions";
import * as busboy from 'busboy';

export async function Test(request: HttpRequest, context: InvocationContext): Promise<HttpResponseInit> {
    try {
        console.log('POST request');

        const formData = {};
        const bb = busboy({ headers: Object.fromEntries(request.headers), limits: { fieldSize: 999999999 } });
        bb.on('field', (name, val, info) => {
            formData[name] = val;
        });
        bb.on('close', () => {
            console.log('Done parsing form!');
        });

        const body = await request.text();
        bb.write(body);
        bb.end();

        let data = formData["jsondatafield"];
        if (data && typeof data == "string") {
            console.log(`Data length: ${data.length}`);
            return { jsonBody: JSON.parse(data) }
        }
        return { status: 400, body: "bad request" }
    } catch (error) {
        return { status: 500, body: error?.message }
    }
};

app.post('Test', {
    handler: Test
});

from azure-functions-nodejs-library.

Dev-InSoft avatar Dev-InSoft commented on June 27, 2024 1

Ok, thank you, I understand then that the problem is due to the use found in "undici", which is why we have decided not to work with the request function:

await request.formData()

But rather with busBoy but working the same with “undici” data types such as "FormData", since in the same function we receive files and fields

import { app, HttpRequest, HttpResponseInit, InvocationContext } from "@azure/functions";
import * as busboy from 'busboy';
import { FormData, FormDataEntryValue } from "undici";
import type { Readable } from "stream";
import type { IncomingHttpHeaders } from "http";

function parseFormData(body: Uint8Array, headers: IncomingHttpHeaders): Promise<FormData> {
	return new Promise<FormData>((resolve, reject) => {
		try {
			let result: FormData = new FormData();
			let files: { [key: string]: Buffer } = {}
			let bb: busboy.Busboy = busboy({ headers, limits: { fieldSize: Number.MAX_SAFE_INTEGER } });
			bb.on('field', (name: string, value: string) => result.append(name, value));
			bb.on('file', (name: string, file: Readable, info: busboy.FileInfo) => {
				if (!files[name]) files[name] = Buffer.from([]);
				file.on('data', (data) => files[name] = Buffer.concat([files[name], data]))
				file.on('end', () => result.append(name, new File([files[name]], info.filename, { type: info.mimeType })))
			})
			bb.on('close', () => resolve(result));
			bb.on('error', (error: unknown) => reject(error))
			bb.write(body);
			bb.end();
		} catch (error) {
			reject(error)
		}
	})
}

export async function Test(request: HttpRequest, context: InvocationContext): Promise<HttpResponseInit> {
	try {
		const formData = await parseFormData(new Uint8Array(await request.arrayBuffer()), Object.fromEntries(request.headers))
		let data: FormDataEntryValue | null = formData.get("jsondatafield")// formData.get("jsondatafield");
		if (data && typeof data == "string")
			return { jsonBody: JSON.parse(data) }
		return { status: 400, body: "bad requet" }
	} catch (error) {
		return { status: 500, body: error?.message }
	}
};

app.post('Test', {
	handler: Test
});

from azure-functions-nodejs-library.

Dev-InSoft avatar Dev-InSoft commented on June 27, 2024

In the following example, it sent a JSON that has a size of 1,197,284 bytes, but when the information arrives at the server, it indicates that the size of the “jsondatafield” field is only 1,048,576 bytes, evidencing that it trims it.

import { app, HttpRequest, HttpResponseInit, InvocationContext } from "@azure/functions";
import type { FormData, FormDataEntryValue } from "undici"

export async function Test(request: HttpRequest, context: InvocationContext): Promise<HttpResponseInit> {
	try {
		//let text = await request.text();
		let formData: FormData = await request.formData();
		let data: FormDataEntryValue | null = formData.get("jsondatafield");
		if (data && typeof data == "string")
			return { jsonBody: JSON.parse(data) }
		return { status: 400, body: "bad requet" }
	} catch (error) {
		return { status: 500, body: error?.message }
	}
};

app.post('Test', {
	handler: Test
});

image

If I read the Body information with await request.text() I notice that the JSON is complete, and that it is truncated by await request.formData()

image

In the same example, if I sent a JSON with a size less than 1,048,576, as in the following case, I am sending one of 1040640 bytes, the reading is normal and there are no problems in the execution of the function

image

Below I attach an example in postman and a mini project in which the case can be reproduced
Azure Function V4.postman.json
Example.zip

from azure-functions-nodejs-library.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.