Giter VIP home page Giter VIP logo

dialoginsight's Introduction

What is dialoginsight?

dialoginsight allows you to export dynamic and detailed dialog profiles from OpenSIPs as Prometheus style metrics.

OpenSIPs Statistics

This application does not export standard OpenSIPs statistics. If are looking to export OpenSIPs statistics it has been supported directly by the prometheus module since version 3.2. For older versions you can use opensips_exporter.

Features

  • Supports OpenSIPs 3.2+ via the mi_http module and JSON-RPC 2.0
  • Export all dialog profiles or only a specific list
  • Ability to use dynamic metric labels on exported metrics from the OpenSIPs script.

Configuration

Configuration is done via a json config file (typically /etc/dialoginsight/config.json) or command line flags.

Configuration Settings

These may be used as command line flags or as fields in the json configuration.

  • listen - Local IP and port for the prometheus exporter to listen on. Metrics are exported under http://listen/metrics (default "127.0.0.1:10337")
  • opensips_mi - URL to the mi_http instance for OpenSIPs. (default "http://127.0.0.1:8888/mi")
  • export_all - Whether or not to export all dialog profiles from the instance. (default "true")
  • export_profiles - List of dialog profiles to export. Used if export_all is set to false. Can also be used with export_all set to true to provide hints about shared and replicated profiles. (Example: [ "myprofile", "sharedprofile/s", "sharedprofile/b" ])
  • insight_label - Dialog value starting prefix to indicate it is an insight value (contains labels to process). (default "insight")
  • timeout - Timeout duration for OpenSIPs API requests. (default "2s")
  • idle_remove - If a metric is idle for this long it will be removed from memory. (default "1m")
  • enable_profiling - Enables access to profiling via http://listen/debug/pprof/ (default "false")

Additional Flags

  • config - Allows specifying the configuration file to use.

Exporting Dialog Profiles

Export Namespaces

Depending on the type of profile they will be exported under either dialoginsight_exported_profile_ (standard OpenSIPs profiles) or dialoginsight_profile_ (extended insight profiles) metric prefixes.

Each profile is exported as its own metric of type dialogs

For example, lets says you have a profile named global that all calls are a part of.

set_dlg_profile("global")

This will be exported as:

dialoginsight_exported_profile_global_dialogs 1

Profiles with Values

Dialog profiles with values will be exported with a label of name 'value' containing the profile value. For example:

set_dlg_profile("customer", "1234")

Will be exported as:

dialoginsight_exported_profile_customer_dialogs{value="1234"} 1

Shared or Replicated Profiles

Currently OpenSIPs reports the name of shared and/or replicated profiles without their tags, but the tags must be included in API calls for them to work correctly. To work around this include the profile name with tags in the export_profiles configuration. This will be used to learn what profiles are shared or replicated and the full name with tags will be used for API calls.

When a profile indicates that it it shared or replicated we will automatically include the shared or replicated labels in the metric with the value of yes. If these labels overlap with insight labels, the insight labels will overwrite them.

Insight

Adding insight to the exported profiles allows you to set dynamic metric labels and increase visibility into active calls.

Insight Values

Insight values are profile values that start with insight_label followed by a : and a series of label=value pairs separated by ;

label must follow Prometheus conventions and match the pattern of '^[a-zA-Z_][a-zA-Z0-9_]*'

  • If a label is not valid the label=value pair will be silently dropped

value may contain any valid UTF-8 character, except ';'

Example format:

insight: cust=1234;carrier=4567;some_stat=5

The format is very similar to a SIP header whose body contains only a list of parameters with values.

Adding Insight

You can can add insight to existing dialog profiles, or use separate profiles.

Lets take our customer example from above

set_dlg_profile("customer", "1234")

Exports as:

dialoginsight_exported_profile_customer_dialogs{value="1234"} 1

Lets add insight to track which source IP the customer is sending from:

set_dlg_profile("customer", "1234")
set_dlg_profile("customer", "insight: cust=1234;scr_ip=$si")

These will be exported as:

dialoginsight_exported_profile_customer_dialogs{value="1234"} 1
dialoginsight_profile_customer_dialogs{cust="1234",src_ip="192.168.168.1"} 1

Notes on Cardinality

dialoginsight makes it very easy to create detailed views of your active call dialogs. However one pitfall that you should be aware of is high cardinality. High cardinality can cause performance issues, both with exporting metrics and querying them. Therefore it is important to be mindful of how many potential metrics you may be exporting.

Lets take the example of:

set_dlg_profile("tracking", "insight: cust=1234;carrier=4567")

If you have 100 customers, and 100 carriers, this can potentially result in 10,000 (100*100) unique combinations. While that is not bad by itself, lets say we decide to also track the destination state for U.S. Domestic calls:

set_dlg_profile("tracking", "insight: cust=1234;carrier=4567;called_state=NY")

We now have a potential for 500,000 (100*100*50) unique combinations. This shows how quickly cardinality can grow.

Ways to reduce cardinality

One way to limit cardinality is to use less combinations of labels in a single metric. If we take the example from above, and instead track the called_state by cust and carrier individually:

set_dlg_profile("tracking", "insight: cust=1234;carrier=4567")
set_dlg_profile("cust_called_state", "insight: cust=1234;called_state=NY")
set_dlg_profile("carrier_called_state", "insight: carrier=4567;called_state=NY")

Our potential unique combinations is greatly reduced to 20,000 ((100*100) + (100*50) + (100*50)). You lose granularity in exchange for reduced cardinality.

Issues or Pull Requests

https://github.com/rrb3942/dialoginsight/issues

License

MIT

Copyright (c) 2023 Ryan Bullock

dialoginsight's People

Contributors

rrb3942 avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar

dialoginsight's Issues

Scraping data for shared or replicated profiles

Hello, thanks for a great little project - @recourse set me upon it and I think it is going to be very useful!

However some of the dialog profile names that I used are suffixed with /b and /s naming flags as per the documentation for marking binary replicated and shared dialog profiles here https://opensips.org/html/docs/modules/3.2.x/dialog.html

I notice that the scraper does indeed request these profiles and a response is received with the correct profile flag set (replicated:yes in this case) but the data does not end up in prometheus.

Check the request (from dialoginfo to opensips) for egress/b below

T 2023/09/12 12:29:48.258981 127.0.0.1:39022 -> 127.0.0.1:8888 [AP] #314
{"jsonrpc": "2.0", "id": "12360", "method": "profile_get_size", "params": ["egress/b"]}

T 2023/09/12 12:29:48.259047 127.0.0.1:8888 -> 127.0.0.1:39022 [AP] #316
HTTP/1.1 200 OK.
Connection: close.
Content-Length: 125.
Content-Type: application/json.
Date: Tue, 12 Sep 2023 11:29:48 GMT.
.
{"jsonrpc":"2.0","result":{"Profile":{"name":"egress","value":null,"count":0,"shared":"no","replicated":"yes"}},"id":"12360"}

Is this something I can solve with configuration somehow? If not, perhaps these could be exported with a naming convention of -b or -s into prometheus?

Thanks

Pete

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.