BUG: Critical error on certain log objects about logtail-js HOT 12 CLOSED

I might be having the same problem. I just signed up for logtail today and am using the winston integration to compare to sematext but some of my logs are not getting to logtail.

I'm getting an error in @msgpack's Encoder.ts:

      Error: Unrecognized object: [object Function]
           at Encoder.encodeObject (/node_modules/@msgpack/msgpack/src/Encoder.ts:200:13)
           at Encoder.doEncode (/node_modules/@msgpack/msgpack/src/Encoder.ts:54:12)
           at Encoder.encodeMap (/node_modules/@msgpack/msgpack/src/Encoder.ts:286:14)
           at Encoder.encodeObject (/node_modules/@msgpack/msgpack/src/Encoder.ts:197:12)
           at Encoder.doEncode (/node_modules/@msgpack/msgpack/src/Encoder.ts:54:12)
           at Encoder.encodeMap (/node_modules/@msgpack/msgpack/src/Encoder.ts:286:14)
           at Encoder.encodeObject (/node_modules/@msgpack/msgpack/src/Encoder.ts:197:12)
           at Encoder.doEncode (/node_modules/@msgpack/msgpack/src/Encoder.ts:54:12)
           at Encoder.encodeArray (/node_modules/@msgpack/msgpack/src/Encoder.ts:242:12)
           at Encoder.encodeObject (/node_modules/@msgpack/msgpack/src/Encoder.ts:193:12)

I'd hope to be able to send arbitrary json to the logger since this works with other logging servers.

from logtail-js.

@adikus thanks for the response - I don't have a PR as my solution has been to add a filter on the winston integration side. I don't know where the best place would be to solve this on your end.

I'd be happy to test if you have a fix though, I can remove my workaround if you have a solution.

My fix is not ideal but looks like this:

const customFormat = winston.format((info, opts) => {
  const keys = Object.keys(info)
  for (const key of keys) {
    if (typeof info[key] === 'object') {
      // Remove functions from this child
      info[key] = JSON.parse(JSON.stringify(info[key], null))
    }
  }
  return info
})

export const logger = winston.createLogger({
  exitOnError: false,
  defaultMeta: {
    app: EnvVars.APP
  },
  format: winston.format.combine(
    customFormat(),
    winston.format.json()
  ),
  // @ts-ignore
  transports: [logseneTransport, logtailTransport]
})

Thanks

from logtail-js.

@sambaker @dawsbot I released a new version 0.1.12 - if you have some time feel free to give it a test. It should strip any values that aren't null, boolean, number, string, Date, Array or Object.

from logtail-js.

@dawsbot, it looks like logging any object with a function property - I can get an example together later and update here. Since I'm using winston and the logtail transport, it's not clear that it's a problem with logtail-js vs the winston integration. I'll see if I can figure out where the problem is - might not be today though.

You can get rid of functions with JSON.parse(JSON.stringify(data,null)). It won't handle circular refs etc but it will remove functions and properties with undefined values (the null replacer passed to JSON.stringify does this by default). That might be a simple fix for you. I'm trying to figure out how to do the same with winston.

from logtail-js.

Hi @dawsbot , @sambaker!
Thanks for reporting and figuring this one out. I'll try to figure out a best way to strip "invalid" / "non acceptable" elements from the logged object.
I'm also curious about the differences between the errors you two are seeing - looks like one failed in msgpack encoding in this library, while the other did go through and was rejected by our ingesting endpoint.

If one you wanted to draft a PR for this I'd be happy to give it a review and merge, but otherwise I'll have a look at this myself sometime this week 🤞

from logtail-js.

Ok, as suggested in the exception, mine has a specific cause - one of my data objects does have a function property. The Sematext (or the Logsene/Sematext integration with Winston) auto-removes, or hides, that property. It would be great if logtail did that too, because there seems to be a set of parseable types that logtail can process and anything else causes exceptions and failed logging.

I can solve it by adding some middleware to recursively remove functions but if there's any other data that might cause exceptions, that would be useful to know. It would be better if logtail just ignored unprocessable properties. @dawsbot if you have a function on your object, that might be the cause of your exceptions. If not, I can open a separate ticket for my issue.

from logtail-js.

@sambaker Great debugging here! We should PR a change to this library which has recursive omitting of all value types that are function.

That would be the first step to fix my error (which mysteriously disappeared right after I submitted this issue)

Here was the TS type for what I believe choked this library:

export interface AxiosError<T = any, D = any> extends Error {
  config: AxiosRequestConfig<D>;
  code?: string;
  request?: any;
  response?: AxiosResponse<T, D>;
  isAxiosError: boolean;
  toJSON: () => object;
}

from logtail-js.

@sambaker do you have a minimum reproducible example of this error?

from logtail-js.

Nice, well I'm not actually blocked by this issue any longer. I was mostly trying to help contribute a fix back to the repo together.

I don't have any say as to what gets merged here anyways, would love to see someone from @logtail jump in with thoughts. @simonrozsival and @JendaTovarys are the two most-recent contributors

from logtail-js.

Yeah, I'd like to get some input from the maintainers at logtail. I'd like them to either handle arbitrary objects, or document what types of field don't throw exceptions and then include examples for sanitizing data reliably. I've also solved it for my case for now.

from logtail-js.

Just emailed @veronikakolejak - she reached out via email when I first became a paying customer. Still am so hopefully they check GitHub once in awhile 🤷

from logtail-js.

Thanks for the response on this @adikus. I cannot recreate the error any longer so I'll leave the PR to someone else.

from logtail-js.