"Drill to Details by" Feature Ignores Applied Filters in Superset 4.0.1 about superset HOT 3 OPEN

The issue you're experiencing with the "Drill to details by" feature not applying filters correctly in Apache Superset version 4.0.1 is recognized and has been addressed in a subsequent update. Specifically, a fix related to enabling the DrillBy feature in charts without filters (dimensions) might also resolve issues related to filter application in drill-down SQL query generation. This fix is documented in the pull request #27941.

The "Drill to details by" feature's behavior, including how it handles filters in SQL query generation, is influenced by several factors, including chart configurations, user permissions, and the specific version of Superset you're using. Known limitations in version 4.0.1, such as database support, aggregations, filters requirement, and chart type support, could also contribute to the problem.

To address this issue, ensure you're running a version of Superset that includes the fix mentioned above. If the problem persists even after updating, it may be worth reviewing the specific configurations and settings related to the "Drill to details by" feature in your Superset installation, including chart type support and filter handling.

If you need further assistance or have additional questions, feel free to ask.

To continue the conversation, mention @dosu.

from superset.

To resolve the issue where the "Drill to details by" feature in Apache Superset version 4.0.1 ignores applied filters, ensure that filters are correctly passed and applied both in the frontend and backend.

Frontend Changes

1. Pass Filters to DrillDetailMenuItems Component:
   Ensure the filters object contains the correct filter values and is passed to the DrillDetailMenuItems component.

   if (showDrillToDetail) {
       menuItems.push(
         <DrillDetailMenuItems
           chartId={id}
           formData={formData}
           filters={filters?.drillToDetail}
           isContextMenu
           contextMenuY={clientY}
           onSelection={onSelection}
           submenuIndex={showCrossFilters ? 2 : 1}
           showModal={drillModalIsOpen}
           setShowModal={setDrillModalIsOpen}
           {...(additionalConfig?.drillToDetail || {})}
         />,
       );
     }

2. Verify handleContextMenu Function:
   Ensure the handleContextMenu function constructs the drillToDetailFilters based on the selected filters.

   const handleContextMenu = useCallback(
       (
         e: MouseEvent,
         colKey: (string | number | boolean)[] | undefined,
         rowKey: (string | number | boolean)[] | undefined,
         dataPoint: { [key: string]: string },
       ) => {
         if (onContextMenu) {
           e.preventDefault();
           e.stopPropagation();
           const drillToDetailFilters: BinaryQueryObjectFilterClause[] = [];
           if (colKey && colKey.length > 1) {
             colKey.forEach((val, i) => {
               const col = cols[i];
               const formatter = dateFormatters[col];
               const formattedVal = formatter?.(val as number) || String(val);
               if (i > 0) {
                 drillToDetailFilters.push({
                   col,
                   op: '==',
                   val,
                   formattedVal,
                   grain: formatter ? timeGrainSqla : undefined,
                 });
               }
             });
           }
           if (rowKey) {
             rowKey.forEach((val, i) => {
               const col = rows[i];
               const formatter = dateFormatters[col];
               const formattedVal = formatter?.(val as number) || String(val);
               drillToDetailFilters.push({
                 col,
                 op: '==',
                 val,
                 formattedVal,
                 grain: formatter ? timeGrainSqla : undefined,
               });
             });
           }
           onContextMenu(e.clientX, e.clientY, {
             drillToDetail: drillToDetailFilters,
             crossFilter: getCrossFilterDataMask(dataPoint),
             drillBy: dataPoint && {
               filters: [
                 {
                   col: Object.keys(dataPoint)[0],
                   op: '==',
                   val: Object.values(dataPoint)[0],
                 },
               ],
               groupbyFieldName: rowKey ? 'groupbyRows' : 'groupbyColumns',
             },
           });
         }
       },
       [
         cols,
         dateFormatters,
         getCrossFilterDataMask,
         onContextMenu,
         rows,
         timeGrainSqla,
       ],
     );

Backend Changes

1. Apply Filters in _get_drill_detail Function:
   Ensure the _get_drill_detail function in superset/common/query_actions.py correctly applies the filters to the query object.

   def _get_drill_detail(
       query_context: QueryContext, query_obj: QueryObject, force_cached: bool = False
   ) -> dict[str, Any]:
       datasource = _get_datasource(query_context, query_obj)
       query_obj = copy.copy(query_obj)
       query_obj.is_timeseries = False
       query_obj.orderby = []
       query_obj.metrics = None
       query_obj.post_processing = []
       qry_obj_cols = []
       for o in datasource.columns:
           if isinstance(o, dict):
               qry_obj_cols.append(o.get("column_name"))
           else:
               qry_obj_cols.append(o.column_name)
       query_obj.columns = qry_obj_cols
       return _get_full(query_context, query_obj, force_cached)

2. Check QueryContext Object:
   Ensure the QueryContext object handles filters correctly when generating SQL queries for drill-down details.

   def _apply_filters(self, query_object: QueryObject) -> None:
       if query_object.time_range:
           for filter_object in query_object.filter:
               if filter_object["op"] == "TEMPORAL_RANGE":
                   filter_object["val"] = query_object.time_range

3. Verify buildQuery Function:
   Ensure the buildQuery function in superset-frontend/src/filters/components/Select/buildQuery.ts correctly adds extra filters based on user input.

   const buildQuery: BuildQuery<PluginFilterSelectQueryFormData> = (
     formData: PluginFilterSelectQueryFormData,
     options,
   ) => {
     const { search, coltypeMap } = options?.ownState || {};
     const { sortAscending, sortMetric } = { ...DEFAULT_FORM_DATA, ...formData };
     return buildQueryContext(formData, baseQueryObject => {
       const { columns = [], filters = [] } = baseQueryObject;
       const extraFilters: QueryObjectFilterClause[] = [];
       if (search) {
         columns.filter(isPhysicalColumn).forEach(column => {
           const label = getColumnLabel(column);
           if (
             coltypeMap[label] === GenericDataType.String ||
             (coltypeMap[label] === GenericDataType.Numeric &&
               !Number.isNaN(Number(search)))
           ) {
             extraFilters.push({
               col: column,
               op: 'ILIKE',
               val: `%${search}%`,
             });
           }
         });
       }
       const sortColumns = sortMetric ? [sortMetric] : columns;
       const query: QueryObject[] = [
         {
           ...baseQueryObject,
           columns,
           metrics: sortMetric ? [sortMetric] : [],
           filters: filters.concat(extraFilters),
           orderby:
             sortMetric || sortAscending !== undefined
               ? sortColumns.map(column => [column, !!sortAscending])
               : [],
         },
       ];
       return query;
     });
   };

By ensuring that filters are correctly passed and applied at each step, you should be able to resolve the issue where the "Drill to details by" feature ignores the applied filters.

To continue the conversation, mention @dosu.

from superset.

Hi ,
This is not related to the #27941 (i've checked the problem statement of this fix)

from superset.