mozilla / metric-config-parser Goto Github PK

See jetstream#1623

As part of the Metrics & Measurement Team's Metrics Taxonomy work, we'd like to request some parsing/checking of fields in metric-hub metrics, specifically:

• the owner field appears to be a single string or a list of strings, I think either is fine, but we should check that via parsing; also, it would be nice to display the owner(s) in metric-hub's interface and datahub (where it already has a field for Owners in the right-hand toolbar, but doesn't seem to be connected to this field)
• the deprecated field appears to be a boolean, but we should check that via parsing; again, it would be very useful to display that prominently in metric-hub's interface and datahub (and even perhaps change other visual aspects of metrics where the value is True to make it very obvious to users that the metric is deprecated and should not be used)
• the category field appears to be a string, and even though it's not used currently (according to docs), it could be nice to display that in metric-hub's interface and datahub (and it would be very cool to be able to click on it to see everything in that category or search by category -- though that may be a more complex request)
• a field that hasn't been used before but we'd like to start using is level, which is a string that can have 1 of 3 possible values ("gold", "silver", "bronze"); this would also be important to surface in metric-hub's interface and datahub
  Lastly, we'll note that all 4 of these fields should be optional.

Some metrics may only be defined for a single branch. These can be confusing when displayed in Experimenter. It would be nice to have the ability to suppress metric display in Experimenter as configured through metric-hub. For example:

[metrics.my_metric]
suppress_experimenter_display = true

I have a metric here defined in a Jetstream Outcome that is frequently used in experiments.

I use mozanalysis to size. I would like to be able to use the definition directly from the Outcome in my sizing scripts. Something like

from mozanalysis.config import ConfigLoader
metric = ConfigLoader.get_metric(metric_slug="urlbar_search_count", app_name="firefox_desktop", outcome="firefox_suggest")

Related to mozilla/mozanalysis#187

I follow the docs here to use a Segment Data Source from Metrics Hub with Mozanalysis, but got a TypeError. I'm not sure what the intended usage is.

Python:

from metric_config_parser.config import ConfigCollection
config_collection = ConfigCollection.from_github_repo("https://github.com/mozilla/metric-hub")
clients_daily = config_collection.get_segment_data_source_definition("clients_daily", app_name="firefox_desktop")

from mozanalysis.segments import Segment
us_clients = Segment(
    name="us_clients",
    data_source=clients_daily,
    select_expr="COALESCE(country = 'US', FALSE)",
)

Error:

TypeError: ("'data_source' must be <class 'mozanalysis.segments.SegmentDataSource'> (got SegmentDataSourceDefinition(name='clients_daily', from_expression='mozdata.telemetry.clients_daily', window_start=0, window_end=0, client_id_column='client_id', submission_date_column='submission_date', default_dataset=None) that is a <class 'metric_config_parser.segment.SegmentDataSourceDefinition'>).", Attribute(name='data_source', default=NOTHING, validator=<instance_of validator for type <class 'mozanalysis.segments.SegmentDataSource'>>, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None), <class 'mozanalysis.segments.SegmentDataSource'>, SegmentDataSourceDefinition(name='clients_daily', from_expression='mozdata.telemetry.clients_daily', window_start=0, window_end=0, client_id_column='client_id', submission_date_column='submission_date', default_dataset=None))

We have some tests that create a Github repository and run through some artificially created commits. The Github user is set to test which also overwrites the actual local Github settings. We should find a way to avoid this

Our current config has python 3.6 as the minimum requirement, but let's bring that up to date with other parts of analysis (jetstream, mozanalysis) and require >=3.10

Metric.data_source is not hashable.

Mozanalysis expects Metric.data_source to be hashable:

• In the power calculator here
• In Experiments here -- will this break Jetstream?

@scholtzan could you help? this is not urgent