Comments (4)
Hi @Dev-InSoft I'm not familiar with @anzp/azure-function-multipart
, but yes request.formData()
is the recommended way to parse form data. Could you perhaps give more detailed repro steps with sample data? Any idea what characters are missing? For example is it from the end of the data or is it maybe a difference in whitespace?
The biggest problem we have with form data right now is we don't support streaming http requests. I don't know if this is your problem, but if you are sending the request with "transfer-encoding" set to "chunked", that could explain it. We hope to support streams soon (tracked by #97).
from azure-functions-nodejs-library.
So it turns out this default limit of "1048576" is set by the busboy npm package. Your original package "@anzp/azure-function-multipart" and "undici" are both using "busboy" as a dependency, but it seems like only the former supports passing the config on to busboy. My current recommendation would be to use "busboy" directly if you need to configure the limits. The other alternative is to file an issue on undici so that we can configure the limits, but the fetch spec for formData doesn't allow any parameters so I'm skeptical we'd get traction.
Here's some rough code that worked for me. It's just a combination of your code and the busboy sample in their readme:
import { app, HttpRequest, HttpResponseInit, InvocationContext } from "@azure/functions";
import * as busboy from 'busboy';
export async function Test(request: HttpRequest, context: InvocationContext): Promise<HttpResponseInit> {
try {
console.log('POST request');
const formData = {};
const bb = busboy({ headers: Object.fromEntries(request.headers), limits: { fieldSize: 999999999 } });
bb.on('field', (name, val, info) => {
formData[name] = val;
});
bb.on('close', () => {
console.log('Done parsing form!');
});
const body = await request.text();
bb.write(body);
bb.end();
let data = formData["jsondatafield"];
if (data && typeof data == "string") {
console.log(`Data length: ${data.length}`);
return { jsonBody: JSON.parse(data) }
}
return { status: 400, body: "bad request" }
} catch (error) {
return { status: 500, body: error?.message }
}
};
app.post('Test', {
handler: Test
});
from azure-functions-nodejs-library.
Ok, thank you, I understand then that the problem is due to the use found in "undici", which is why we have decided not to work with the request function:
await request.formData()
But rather with busBoy but working the same with “undici” data types such as "FormData", since in the same function we receive files and fields
import { app, HttpRequest, HttpResponseInit, InvocationContext } from "@azure/functions";
import * as busboy from 'busboy';
import { FormData, FormDataEntryValue } from "undici";
import type { Readable } from "stream";
import type { IncomingHttpHeaders } from "http";
function parseFormData(body: Uint8Array, headers: IncomingHttpHeaders): Promise<FormData> {
return new Promise<FormData>((resolve, reject) => {
try {
let result: FormData = new FormData();
let files: { [key: string]: Buffer } = {}
let bb: busboy.Busboy = busboy({ headers, limits: { fieldSize: Number.MAX_SAFE_INTEGER } });
bb.on('field', (name: string, value: string) => result.append(name, value));
bb.on('file', (name: string, file: Readable, info: busboy.FileInfo) => {
if (!files[name]) files[name] = Buffer.from([]);
file.on('data', (data) => files[name] = Buffer.concat([files[name], data]))
file.on('end', () => result.append(name, new File([files[name]], info.filename, { type: info.mimeType })))
})
bb.on('close', () => resolve(result));
bb.on('error', (error: unknown) => reject(error))
bb.write(body);
bb.end();
} catch (error) {
reject(error)
}
})
}
export async function Test(request: HttpRequest, context: InvocationContext): Promise<HttpResponseInit> {
try {
const formData = await parseFormData(new Uint8Array(await request.arrayBuffer()), Object.fromEntries(request.headers))
let data: FormDataEntryValue | null = formData.get("jsondatafield")// formData.get("jsondatafield");
if (data && typeof data == "string")
return { jsonBody: JSON.parse(data) }
return { status: 400, body: "bad requet" }
} catch (error) {
return { status: 500, body: error?.message }
}
};
app.post('Test', {
handler: Test
});
from azure-functions-nodejs-library.
In the following example, it sent a JSON that has a size of 1,197,284 bytes, but when the information arrives at the server, it indicates that the size of the “jsondatafield” field is only 1,048,576 bytes, evidencing that it trims it.
import { app, HttpRequest, HttpResponseInit, InvocationContext } from "@azure/functions";
import type { FormData, FormDataEntryValue } from "undici"
export async function Test(request: HttpRequest, context: InvocationContext): Promise<HttpResponseInit> {
try {
//let text = await request.text();
let formData: FormData = await request.formData();
let data: FormDataEntryValue | null = formData.get("jsondatafield");
if (data && typeof data == "string")
return { jsonBody: JSON.parse(data) }
return { status: 400, body: "bad requet" }
} catch (error) {
return { status: 500, body: error?.message }
}
};
app.post('Test', {
handler: Test
});
If I read the Body information with await request.text() I notice that the JSON is complete, and that it is truncated by await request.formData()
In the same example, if I sent a JSON with a size less than 1,048,576, as in the following case, I am sending one of 1040640 bytes, the reading is normal and there are no problems in the execution of the function
Below I attach an example in postman and a mini project in which the case can be reproduced
Azure Function V4.postman.json
Example.zip
from azure-functions-nodejs-library.
Related Issues (20)
- Multiple Routes for one function (Azure Function App Node v4) HOT 2
- High memory usage when posting a ~77mb payload to a HTTP trigger. HOT 4
- app.<http_verb>() methods don't generate a unique id for the same route name but different verbs HOT 1
- Support open telemetry
- Debugging in VS code terminates when hot reload is running `npm run watch` HOT 4
- Support OpenAI extension
- Typescript function app tests using supertest HOT 4
- Table trigger with {id} isn't working HOT 1
- Bearer authorization token is being trucanted. HOT 2
- Support Node 22
- Functions not found after migrating to model v.4 HOT 3
- Build compiler Esbuild support HOT 2
- HTTP 500: Error in logs says Worker was unable to load function: 'Cannot find module 'uuid' HOT 3
- How to support monorepo HOT 18
- Allow for conditional triggering of input bindings HOT 3
- missing disabled functionality for azure functions v4 HOT 3
- Officially support TypeScript 5
- enableHttpStream for nextjs apps HOT 2
- Aborting an Azure Function HOT 2
- Using WEBSITE_RUN_FROM_PACKAGE = URL results in 0 functions found using Azure Functions V4 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from azure-functions-nodejs-library.