#Management of Monitoring Dashboards

#Function Overview

The platform provides powerful dashboard management functionality designed to replace traditional Grafana tools, offering users a more comprehensive and flexible monitoring experience. This feature aggregates various monitoring data from within the platform, presenting a unified monitoring view that significantly enhances your configuration efficiency.

#Main Features

• Supports configuring custom monitoring dashboards for both business views and platform views.
• Enables viewing publicly shared dashboards configured in platform views from business views, with data isolated based on the namespace to which the business belongs.
• Supports managing pannels within the dashboard, allowing users to add, delete, modify pannels, zoom in/out pannels, and move pannels through drag-and-drop.
• Allows setting custom variables within the dashboard for filtering query data.
• Supports configuring groups within the dashboard for managing the pannels. Groups can be displayed repeatedly based on custom variables.
• Supported pannel types include: trend、step line chart、bar chart、horiazontal bar chart、bar gauge chart、gauge chart、table、stat chart、XY chart、pie chart、text.
• One-click import feature for Grafana dashboards.

#Advantages

• Supports user-customized monitoring scenarios without being constrained by predefined templates, truly achieving a personalized monitoring experience.
• Provides a rich array of visualization options, including line charts, bar charts, pie charts, and flexible layout and styling options.
• Integrates seamlessly with the platform's role permissions, allowing business views to define their own monitoring dashboards while ensuring data isolation.
• Deep integration with various functionalities of the container platform, enabling instant access to monitoring data for containers, networks, storage, etc., providing users with comprehensive performance observation and fault diagnosis.
• Fully compatible with Grafana dashboard JSON, allowing easy migration from Grafana for continued use.

#Use Cases

• IT Operations Management: As part of the IT operations team, you can use the monitoring dashboards to unify the display and management of various performance metrics of the container platform, such as CPU, memory, network traffic, etc. By customizing monitoring reports and alert rules, you can promptly detect and pinpoint system issues, enhancing operational efficiency.
• Application Performance Analysis: For application developers and testers, monitoring dashboards offer various rich visualization options to intuitively display application running states and resource consumption. You can customize dedicated monitoring views tailored to different application scenarios to deeply analyze application performance bottlenecks and provide a basis for optimization.
• Multi-Cluster Management: For users managing multiple container clusters, monitoring dashboards can aggregate monitoring data from disparate clusters, allowing you to grasp the overall operational state of the system at a glance.
• Fault Diagnosis: When a system issue occurs, monitoring dashboards provide you with comprehensive performance data and analytical tools to quickly pinpoint the root cause of the problem. You can swiftly view fluctuations in relevant monitoring metrics based on alert information for in-depth fault analysis.

#Prerequisites

Currently, monitoring dashboards only support viewing monitoring data collected by monitoring components installed in the platform. Therefore, you should prepare as follows before configuring a monitoring dashboard:

• Ensure that the cluster for which you want to configure the monitoring dashboard has monitoring components installed, specifically the ACP Monitor with Prometheus or ACP Monitor with VictoriaMetrics plugin.
• Ensure that the data you wish to display on the dashboard has been collected by the monitoring components.

#Relationship Between Monitoring Dashboards and Monitoring Components

• Monitoring dashboard resources are stored in the Kubernetes cluster. You can switch views between different clusters using the Cluster tab at the top.
• Monitoring dashboards depend on the monitoring components in the cluster for querying data sources. Therefore, before using monitoring dashboards, ensure that the current cluster has successfully installed monitoring components and that they are operating normally.
• The monitoring dashboard will default to requesting monitoring data from the corresponding cluster. If you install the VictoriaMetrics plugin in proxy mode in the cluster, we will request the storage cluster for you to query the corresponding data for this cluster without the need for special configuration.

#Manage Dashboards

A dashboard is a collection composed of one or more pannels, organized and arranged in one or more rows to provide a clear view of relevant information. These pannels can query raw data from data sources and transform it into a series of visual effects supported by the platform.

#Create a Dashboard

1. Click Create Dashboard, reference the following instructions to configure relevant parameters.

2. After adding, click Create to finish creating the dashboard. Next, you need to add variables, add pannels, and add groups to complete the overall layout design.

#Import Dashboard

The platform supports direct import of Grafana JSON to convert it into a monitoring dashboard for display.

• Currently, only Grafana JSON of version V8+ is supported; lower versions will be prohibited from being imported.
• If any pannels within the imported dashboard are not within the platform's supported scope, they may be displayed as unsupported pannel types, but you can modify the pannel's settings to achieve normal display.
• After importing the dashboard, you can perform any management actions as usual, which will not differ from pannels created in the platform.

#Add Variables

1. In the variable form area, click Add.

   Parameter	Description
   Type	Currently only supports Query type variables, which allow you to filter data based on the feature dimensions of time series. The query expression can be specified to dynamically calculate and generate query results.
   Display Filter	Default value for displaying drop-down filter options on the dashboard homepage; supports showing the name and value, value only, or no display (hiding the filter box).
   Query Settings	Using Query type variables allows you to filter data based on the feature dimensions of time series. When defining query settings, besides using PromQL to query time series, the platform also provides some common variables and functions. Reference Common Functions and Variables.
   Regular Expression	By using regular expressions, you can filter out the desired values from the content returned by the variable queries. This makes each option name in the variable more expected. You can preview if the filtered values meet expectations in Variable Value Preview.
   Selection Settings	- Multiple Selection: When selected from the top filters on the dashboard homepage, allows the selection of multiple options simultaneously. You need to reference this variable in the query expression of the pannels to view the data corresponding to the variable value. - All: If checked, an option containing All will be enabled in the filter options to select all variable data.

2. Click OK to add one or more variables.

#Add Pannels

Add multiple pannels to the currently created monitoring dashboard to display data information for different resources.

Tip: You can customize the size of a pannel by clicking the lower right corner; click anywhere on the pannel to rearrange the order of the pannels.

1. Click Add Pannel, reference the following instructions to configure relevant parameters.

• Pannel Preview: The area will dynamically display the data information corresponding to the added metrics.

• Add Metric: Configure the pannel title and monitoring metrics in this area.

• Adding Method: Supports using built-in metrics or using natively customized metrics. Both methods will take the union and be effective simultaneously.

  • Built-in Metrics: Select commonly used metrics and legend parameters built into the platform to display the data information under the current pannel.
    • Note: All metrics added to the pannel must have a unified unit; it is not possible to add metrics with multiple units to one pannel.
  • Native: Customize the metric unit, metric expression, and legend parameters. The metric expression follows PromQL syntax; for details, please refer to PromQL Official Documentation.

• Legend Parameters: Control the names corresponding to the curves in the pannels. Text or templates can be used:

  • Rule: The input value must be in the format {{.xxxx}}; for example, {{.hostname}} will replace it with the value corresponding to the hostname label returned by the expression.
  • Tip: If you input an incorrectly formatted legend parameter, the names corresponding to the curves in the pannel will be displayed in their original format.

• Instant Switch: When the Instant switch is turned on, it will query instant values through Prometheus's Query interface and sort them, as in statistical charts and gauge charts. If off, it will use the query_range method to calculate, querying a series of data over a specific time period.

• Pannel Settings: Supports selecting different pannel types for visualizing metric data. Please refer to Manage Pannels.

2. Click Save to complete adding the pannels.

3. You can add one or more pannels within the dashboard.

4. After adding the pannels, you can use the following operations to ensure the display and size of the pannels meet your expectations.

   • Click the lower right corner of the pannel to customize its size.
   • Click anywhere on the pannel to rearrange the order of the pannels.

5. After adjusting, click the Save button on the dashboard page to save your modifications.

#Add Groups

Groups are logical dividers within the dashboard that can group pannels together.

1. Click the Add Pannel drop-down menu > Add Group, and reference the following instructions to configure relevant parameters.

• Group: The name of the group.
• Repeat: Supports disabling repeats or selecting variables for the current pannels.
  • Disable Repeat: Do not select a variable, and use the default created group.
  • Parameter Variables: Select the variables created in the current pannels, and the monitoring dashboard will generate a row of identical sub-groups for each corresponding value of the variable. Sub-groups do not support modifications, deletions, or moving of the pannels.

2. After adding the group, you can perform the following operations on the group to manage the pannel display within the dashboard.

   • Groups can be collapsed or expanded to hide part of the content in the dashboard. Pannels within collapsed groups will not send queries.

   • Move the pannel into the group to allow that pannel to be managed by that group. The group will manage all pannels between it and the next group.

   • When a group is folded, you can also move all pannels managed by that group together.

   • The folding and unfolding of groups also constitutes an adjustment to the dashboard. If you want to maintain this state when reopening this dashboard next time, please click the Save button.

#Switch Dashboards

Set the created custom monitoring dashboard as the main dashboard. When re-entering the monitoring dashboard feature, the main dashboard data will be displayed by default.

1. In the left navigation bar, click Operations Center > Monitoring > Monitoring Dashboards.

2. By default, the main monitoring dashboard is entered. Click Switch Dashboard.

3. You can find dashboards by filtering through labels or searching by name, and switch main dashboards via the Main Dashboard switch.

#Other Operations

You can click the operation button on the right side of the dashboard page to perform actions on the dashboard as needed.

#Manage Pannels

The platform provides various visualization methods to support different use cases. This chapter will mainly introduce these pannel types, configuration options, and usage methods.

#Pannel Description

#Pannel Configuration Description

#General Parameters

#Special Parameters for Pannels

#Create Monitoring Dashboards via CLI

1. Create a new YAML configuration file named example-dashboard.yaml.
2. Add the MonitorDashboard resource to the YAML file and submit it. The following example creates a monitoring dashboard named demo-v2-dashborad1:

kind: MonitorDashboard
apiVersion: ait.alauda.io/v1alpha2
metadata:
  annotations:
    cpaas.io/dashboard.version: "3"
    cpaas.io/description: '{"zh":"描述信息","en":""}' # Description field
    cpaas.io/operator: admin
  labels:
    cpaas.io/dashboard.folder: demo-v2-folder1 # Folder
    cpaas.io/dashboard.is.home.dashboard: "False" # Is it the main dashboard?
  name: demo-v2-dashborad1 # Name
  namespace: cpaas-system  # Namespace (all management view creations will occur in this ns)
spec:
  body: # All information fields
    titleZh: 更新显示名称 # Built-in field for Chinese display name (this field is created under the Chinese language)
    title: english_display_name # Built-in field for English display name (this field is created under the English language) Built-in dashboards can set bilingual translations.
    templating: # Custom variables
      list:
        - hide: 0 # 0 means not hidden; 1 means only the label is hidden; 2 means both label and value are hidden
          label: 集群 # Built-in variable display name (label is set to the appropriate name based on the language, e.g., cluster in English)
          name: cluster # Built-in variable name (unique)
          options: # Define dropdown options; if a query retrieves data, it will use requested data; otherwise, it will use options. A default value can be set (generally only used for setting default values)
            - selected: false # Whether to default select
              text: global
              value: global
          type: custom # Custom variable type; currently, only built-in (custom) and query are supported (Importing Grafana will support constant custom interval (after import, it will be changed to a custom variable and will not support auto))
 
        - allValue: "" # Select all, passing options with the format xxx|xxx|xxx; can set allValue for conversion (Grafana retrieves all data for the current variable as xxx|xxx|xxx, adjustments will ensure consistency)
          current: null # Current value of the variable; if not set, defaults to the first in the list
          definition: query_result(kube_namespace_labels) # Query expression for data retrieval
          hide: 0 # 0 means not hidden; 1 means only the label is hidden; 2 means both label and value are hidden
          includeAll: true # Whether to select all
          label: ns # Built-in variable display name
          multi: true # Whether multiple selections are allowed
          name: ns # Variable name (unique)
          options: [] 
          query: ""
          regex: /.*namespace=\"(.*?)\".*/ # Regex expression for extracting variable values
          sort: 2 # Sorting: 1 - ascending alphabetical order; 2 - descending alphabetical order (only these two support temporarily); 3 - ascending numerical order; 4 - descending numerical order
          type: query # Custom variable type
    time: # Dashboard time
      from: now-30m # Start time
      to: now # End time
    repeat: '' # Row repeat configuration; chooses custom variable
    collapsed: 'false' # Row collapsed or expanded configuration
    description: "123" # Description (tooltip after title)
    targets: # Data sources
      - indicator: cluster.node.ready # Metric
        expr: sum (cpaas_pod_number{cluster=\"\"}>0) # PromQL expression
        instant: false # Query mode true retrieves data at a specific time
        legendFormat: "" # Legend
        range: true # Default querying range when retrieving data
        refId: 指标1 # Unique identifier for display name of data source
    gridPos: # Information on the dashboard's positional layout
        h: 8 # Height
        w: 12 # Width (width corresponds to 24 grid units)
        x: 0 # Horizontal position
        y: 0 # Vertical position
    panels: # Pannel data
      title: 图表标题tab # Pannel name
      type: table # Pannel type; currently supports timeseries, barchart, stat, gauge, table, bargauge, row, text, pie (step chart, scatter plot, bar chart, configurable through drawStyle attribute)
      id: a2239830-492f-4d27-98f3-cb7ecb77c56f # Unique identifier
      links: # Links
        - targetBlank: true # Open in a new tab
          title: "1" # Name
          url: "1" # URL address
      transformations: # Data transformations
        - id: "organize" # Type organize; used for sorting, rearranging order, showing fields, whether to display 
          options:
            excludeByName: # Hidden fields
              cluster_cpu_utilization: true
            indexByName: # Sort
                cluster_cpu_utilization: 0,
                Time: 1
            renameByName: # Rename
              Time: ""
              cluster_cpu_utilization: "222"
        - id: "merge" # Merging data
          options:
      fieldConfig: # For defining pannel properties and appearance
        defaults: # Default configuration
          custom: # Custom graphic attributes
            align: "left" # Table alignment: left, center, right
            cellOptions: # Table threshold configuration
              type: color-text # Only supports text for threshold color settings
            spanNulls: false # true connects null values; false does not connect; number == 0 connects null values according to 0
            drawStyle: line # Pannel types: line, bars for bar charts, points for point charts
            fillOpacity: 20 # Exists when drawStyle is area (currently does not support configuration, area defaults to 20)
            thresholdsStyle: # Configures how to display thresholds (currently only supports line)
              mode: line # Threshold display format (area not supported currently)
            lineInterpolation: 'stepBefore' # Step chart configuration; defaults to only supporting stepBefore (stepAfter will be supported later)
          decimals: 3 # Decimal points
          min: 0 # Minimum value (currently not supported for page configuration, only supports imports that have been adapted)
          max: 1 # Maximum value (page configuration only applies to stat gauge barGauge pie)
          unit: "%" # Unit
          mappings: # Value mapping configuration (currently only supports value and range types; special types supported on data)
            - options: # Value mapping rules
                "1": # Corresponding value
                  index: 0
                  text: "Running" # Displayed as Running when value is 1
              type: value # Value mapping type
            - options: # Range mapping rules
                from: 2 # Range start value
                to: 3 # Range end value
                result: # Mapping result
                  index: 1
                  text: "Error" # Values from 2 to 3 will display as Error
              type: range # Mapping type for range
            - type: special # Mapping type for special scenarios
              options:
                match: null # nan null null+nan empty true false
                result:
                  text: xxx
                  index: 2
          thresholds: # Threshold configuration
            mode: absolute # Threshold configuration mode, absolute value mode (currently only supports absolute and percentage mode; percentage mode is not supported yet)
            steps: # Threshold steps
              - color: "#a7772f" # Threshold color
                value: "2" # Threshold value
              - color: "#007AF5" # Default value with no value is the Base
        overrides: # Override configuration
          - matcher:
              id: byName # Match based on field name
              options: node # Corresponding name
            properties: # Override configuration; id currently only supports displayName unit
              - id: displayName # Display name override
                value: "1" # Overridden display name
              - id: unit # Unit override
                value: GB/s # Unit value
              - id: noValue # No value display
                value: No value display
      options:
        orientation: horizontal # Control the layout direction of pannels; applies to gauge and barGauge (stat will be supported later)
        legend: # Legend configuration
          calcs: # Calculating methods (only displays when the legend position is on the right)
            - latest # Currently only supports most recent value
          placement: right # Legend position (right or bottom; defaults to bottom)
          placementRightTop: "" # Configuration for the upper right
          showLegend: true # Whether to display the legend
        tooltip: # Tooltips
          mode: multi # Mode dual selection (only multi-mode supported) All data displayed when the mouse hovers over
          sort: asc # Sorting: asc or desc
        reduceOptions: # Value calculating method (used for aggregating data)
          calcs: # Calculating methods (latest, minimum, maximum, average, sum)
            - latest
          limit: 3 # Pie limits the number of slices
        textMode: 'value' # Stat configuration; defines style for displaying metric value; options are auto, value, value_and_name, name, none (currently not supported in the page configuration, but supported in imports)
        colorMode: 'value' # Stat configuration; defines color mode for displaying metric values; options are none, value, background (defaults to value; not supported in configuration but adapted in import)
        displayLabels: ['name','value', 'percent'] # Fields displayed in pie chart labels
        pieType: "pie" # Pie chart type; options are pie and donut
        mode: 'html' # Text chart type mode; options are html and richText
        content: '<div>xxx</div>' # Content for text chart type
        footer:
          enablePagination: true # Table pagination enabled

#Common Functions and Variables

#Common Functions

When defining query settings, besides using PromQL to set queries, the platform provides some common functions as follows for your reference in customizing query settings.

#Common Variables

While defining query settings, you can combine common functions into variables to quickly define custom variables. Here are some common variable definitions available for your reference:

#Variable Use Case One

Using the query_result(query) function to query the value: node_load5, and extract the IP.

1. In Query Settings, fill in query_result(node_load5).

2. In the Variable Value Preview area, the preview example is node_load5{container="node-exporter",endpoint="metrics",host_ip="192.168.178.182",instance="192.168.178.182:9100"}.

3. In Regular Expression, fill in /.*instance="(.*?):.*/ to filter the value.

4. In the Variable Value Preview area, the preview example is 192.168.176.163.

#Variable Use Case Two

1. Add the first variable: namespace, using the query_result(query) function to query the value: kube_namespace_labels, and extract the namespace.

   • Query Settings: query_result(kube_namespace_labels).

   • Variable Value Preview: kube_namespace_labels{container="exporter-kube-state", endpoint="kube-state-metrics", instance="12.3.188.121:8080", job="kube-state", label_cpaas_io_project="cpaas-system", namespace="cert-manager", pod="kube-prometheus-exporter-kube-state-55bb6bc67f-lpgtx", project="cpaas-system", service="kube-prometheus-exporter-kube-state"}.

   • Regular Expression: /.+namespace=\"(.*?)\".*/.

   • In the Variable Value Preview area, the preview example includes multiple namespaces such as argocd, cpaas-system, and more.

2. Add the second variable: deployment, and reference the variable created earlier:

   • Query Settings: kube_deployment_spec_replicas{namespace=~"$namespace"}.

   • Regular Expression: /.+deployment="(.*?)",.*/.

3. Add a pannel to the current dashboard and reference the previously added variables, for example:

   • Metric Name: pod Memory Usage under Compute Components.

   • Key-Value Pair: kind: Deployment, name: $deployment, namespace: $namespace.

4. Once you have added the pannels and saved them, you can view the corresponding pannel information on the dashboard homepage.

#Notes When Using Built-in Metrics

• workload.cpu.utilization

• workload.memory.utilization

• workload.network.receive.bytes.rate

• workload.network.transmit.bytes.rate

• workload.gpu.utilization

• workload.gpu.memory.utilization

• workload.vgpu.utilization

• workload.vgpu.memory.utilization