Open-metrics endpoint collector for ONTAP

Overview

NetApp Harvest 2.0

The swiss-army knife for monitoring datacenters. The default package collects performance, capacity and hardware metrics from ONTAP clusters. New metrics can be collected by editing the config files. Metrics can be delivered to Prometheus and InfluxDB databases - and displayed in Grafana dashboards.

Harvest's architecture is flexible in how it collects, augments, and exports data. Think of it as a framework for running collectors and exporters concurrently. You are more than welcome to contribute your own collector, plugin or exporter (start with our ARCHITECTURE.md).

Requirements

Harvest is written in Go, which means it runs on recent Linux systems. It also runs on Macs, but the process isn't as smooth yet.

Optional prerequisites:

  • dialog or whiptail (used by the config utility)
  • openssl (used by config)

Hardware requirements depend on how many clusters you monitor and the number of metrics you chose to collect. With the default configuration, when monitoring 10 clusters, we recommend:

  • CPU: 2 cores
  • Memory: 1 GB
  • Disk: 500 MB (mostly used by log files)

Harvest is compatible with:

  • Prometheus: 2.24 or higher
  • InfluxDB: v2
  • Grafana: 7.4.2 or higher

Installation / Upgrade

We provide pre-compiled binaries for Linux, RPMs, and Debs.

Pre-compiled Binaries

Visit the Releases page and copy the tar.gz link you want to download. For example, to download the v21.05.1 release:

wget https://github.com/NetApp/harvest/releases/download/v21.05.1/harvest-21.05.1-1.tar.gz
tar -xvf harvest-21.05.1-1.tar.gz
cd harvest-21.05.1-1

# Run Harvest with the default unix localhost collector
bin/harvest start

If you don't have wget installed, you can use curl like so:

curl -L -O https://github.com/NetApp/harvest/releases/download/v21.05.1/harvest-21.05.1-1.tar.gz

Redhat

Installation of the Harvest package may require root or administrator privileges

Download the latest rpm of Harvest from the releases tab and install with yum.

  $ sudo yum install harvest.XXX.rpm

Once the installation has finished, edit the harvest.yml configuration file located in /opt/harvest/harvest.yml

After editing /opt/harvest/harvest.yml, manage Harvest with systemctl start|stop|restart harvest

Changes install makes

  • Directories /var/log/harvest/ and /var/log/run/ are created
  • A harvest user and group are created and the installed files are chowned to harvest
  • Systemd /etc/systemd/system/harvest.service file is created and enabled

Debian

Installation of the Harvest package may require root or administrator privileges

Download the latest deb of Harvest from the releases tab and install with apt.

  $ sudo apt install ./harvest-<RELEASE>.amd64.deb

Once the installation has finished, edit the harvest.yml configuration file located in /opt/harvest/harvest.yml

After editing /opt/harvest/harvest.yml, manage Harvest with systemctl start|stop|restart harvest

Changes install makes

  • Directories /var/log/harvest/ and /var/log/run/ are created
  • A harvest user and group are created and the installed files are chowned to harvest
  • Systemd /etc/systemd/system/harvest.service file is created and enabled

Docker

Work in progress. Coming soon

Building from source

To build Harvest from source code, first make sure you have a working Go environment with version 1.15 or greater installed. You'll also need an Internet connection to install go dependencies. If you need to build from an air-gapped machine, use go mod vendor from an Internet connected machine first, and then copy the vendor directory to the air-gapped machine.

Clone the repo and build everything.

git clone https://github.com/NetApp/harvest.git
cd harvest
make
bin/harvest version

If you're building on a Mac use GOOS=darwin make build

Checkout the Makefile for other targets of interest.

Quick start

1. Configuration file

Harvest's configuration information is defined in harvest.yml. There are a few ways to tell Harvest how to load this file:

  • If you don't use the --config flag, the harvest.yml file located in the current working directory will be used

  • If you specify the --config flag like so harvest status --config /opt/harvest/harvest.yml, Harvest will use that file

To start collecting metrics, you need to define at least one poller and one exporter in your configuration file. The default configuration comes with a pre-configured poller named unix which collects metrics from the local system. This is useful if you want to monitor resource usage by Harvest and serves as a good example. Feel free to delete it if you want.

The next step is to add pollers for your ONTAP clusters in the Pollers section of the configuration file. Refer to the Harvest Configuration Section for more details.

2. Start Harvest

Start all Harvest pollers as daemons:

$ bin/harvest start

Or start a specific poller(s):

$ bin/harvest start jamaica grenada

Replace jamaica and grenada with the poller names you defined in harvest.yml. The logs of each poller can be found in /var/log/harvest/.

3. Import Grafana dashboards

The Grafana dashboards are located in the $HARVEST_HOME/grafana directory. You can manually import the dashboards or use the harvest grafana command (more documentation).

Note: the current dashboards specify Prometheus as the datasource. If you use the InfluxDB exporter, you will need to create your own dashboards.

4. Verify the metrics

If you use a Prometheus Exporter, open a browser and navigate to http://0.0.0.0:12990/ (replace 12990 with the port number of your poller). This is the Harvest created HTTP end-point for your Prometheus exporter. This page provides a real-time generated list of running collectors and names of exported metrics.

The metric data that's exposed for Prometheus to scrap is available at http://0.0.0.0:12990/metrics/. For more help on how to configure Prometheus DB, see the Prometheus exporter documentation.

If you can't access the URL, check the logs of your pollers. These are located in /var/log/harvest/.

Harvest Configuration

The main configuration file, harvest.yml, consists of the following sections, described below:

Pollers

All pollers are defined in harvest.yml, the main configuration file of Harvest, under the section Pollers.

parameter type description default
Poller name (header) required poller name, user-defined value
datacenter required datacenter name, user-defined value
addr required by some collectors IPv4 or FQDN of the target system
collectors required list of collectors to run for this poller
exporters required list of exporter names from the Exporters section. Note: this should be the name of the exporter (e.g. prometheus1), not the value of the exporter key (e.g. Prometheus)
auth_style required by Zapi* collectors either basic_auth or certificate_auth basic_auth
username, password required if auth_style is basic_auth
ssl_cert, ssl_key optional if auth_style is certificate_auth Absolute paths to SSL (client) certificate and key used to authenticate with the target system.

If not provided, the poller will look for <hostname>.key and <hostname>.pem in $HARVEST_HOME/cert/.

To create certificates for ONTAP systems, see the Zapi documentation
use_insecure_tls optional, bool If true, disable TLS verification when connecting to ONTAP cluster false
log_max_bytes Maximum size of the log file before it will be rotated 10000000 (10 mb)
log_max_files Number of rotated log files to keep 10

Defaults

This section is optional. If there are parameters identical for all your pollers (e.g. datacenter, authentication method, login preferences), they can be grouped under this section. The poller section will be checked first and if the values aren't found there, the defaults will be consulted.

Exporters

All exporters need two types of parameters:

  • exporter parameters - defined in harvest.yml under Exporters section
  • export_options - these options are defined in the Matrix datastructure that is emitted from collectors and plugins

The following two parameters are required for all exporters:

parameter type description default
Exporter name (header) required Name of the exporter instance, this is a user-defined value
exporter required Name of the exporter class (e.g. Prometheus, InfluxDB, Http) - these can be found under the cmd/exporters/ directory

Note: when we talk about the Prometheus Exporter or InfluxDB Exporter, we mean the Harvest modules that send the data to a database, NOT the names used to refer to the actual databases.

Prometheus Exporter

InfluxDB Exporter

Tools

This section is optional. You can uncomment the grafana_api_token key and add your Grafana API token so harvest does not prompt you for the key when importing dashboards.

Tools:
  #grafana_api_token: 'aaa-bbb-ccc-ddd'

Configuring collectors

Collectors are configured by their own configuration files, which are subdirectories in conf/. Each collector can define its own set of parameters.

Zapi

ZapiPerf

Unix

Issues
  • Pollers aren't running on fresh Docker install

    Pollers aren't running on fresh Docker install

    Describe the bug Pollers aren't running

    Environment Provide accurate information about the environment to help us reproduce the issue.

    • Harvest version: [copy paste output of harvest --version] harvest version 21.08.0-6 (commit 485d191) (build date 2021-08-31T11:51:03-0400) linux/amd64
    • Command line arguments used: [e.g. bin/harvest start --config=foo.yml --collectors Zapi] started docker per the easy-start page
    • OS: [e.g. RHEL 7.6, Ubuntu 21.04, Docker 19.03.1-CE] RedHat 8.4
    • Install method: [yum, rhel, native, docker] docker
    • ONTAP Version: [e.g. 9.7, 9.8, 9.9, 7mode] 9.7P7
    • Other:

    To Reproduce Installed\setup to use docker. Started the containers per the docs, I can see them download & start but the pollers never report in. checking them with ./bin/harvest status reports that the pollers are there but not running.

    Expected behavior pollers would be running, I would see my datacenters & clusters in Grafana

    Actual behavior pollers are not running, only "local" listed in Grafana

    Possible solution, workaround, fix Tell us if there is a workaround or fix

    Additional context Add any other context about the problem here. Share logs, error output, etc.

    customer 
    opened by survive-wustl 38
  • Qtree 7mode Template is not available in this repository

    Qtree 7mode Template is not available in this repository

    Hi, I did not find qtree 7mode template. Should I build one myself with qtree-list api? Or there is one built already but this repository is not updated?

    feature status/done 
    opened by jmg011 26
  • Storage type metrics is missing

    Storage type metrics is missing

    Hi, We saw that there is not option to pull metrics about disk type(HHD,SSD). Can you add a metric that will gather info about the disks and output the size the type and etc.

    feature status/done 
    opened by bengoldenberg 26
  • Making dashboards compatible with grafana-API

    Making dashboards compatible with grafana-API

    We want to import grafana-dashboards with Grafana own API. Since we make some specific modification on our dashboard (own variables for region like in the screenshot for example) it would be nice to support the own Grafana-API as well.

    From my current perspective it should be only needed to add some specific identifiers to get them compatible (importable): URL

    Expected behavior Grafana Dashboard should be importable via grafana-API

    Actual behavior own modifications to json-files are needed - afterwards dashboard is empty (not sure if this is maybe a self-made error)

    opened by florianmulatz 19
  • Harvest should collect environment sensor info

    Harvest should collect environment sensor info

    Harvest doesnt collect any chassis sensor info

    Describe the solution you'd like Harvest doesnt collect any chassis sensor info, there are a few important scenarios where having this information is really useful, especially NVMEM battery status which when failed causes a shutdown of a cluster in 24hrs.

    Describe alternatives you've considered N/A

    Additional context This information is available from the environment-sensors-get-iter api, should be fairly easy to map values.

    Example data

    critical-low-threshold  : 1086
    node-name               : node
    sensor-name             : PVDDQ DDR4 AB
    sensor-type             : voltage
    threshold-sensor-state  : normal
    threshold-sensor-value  : 1228
    value-units             : mV
    warning-high-threshold  : 1299
    warning-low-threshold   : 1100
    
    feature status/done 
    opened by hashi825 19
  • Enhance systemd service implementation to monitor all pollers

    Enhance systemd service implementation to monitor all pollers

    Is your feature request related to a problem? Please describe. In the current way systemd service is implemented, harvest just forks all poller child processes and closes. Systemd has no concept of keeping track of all the children. If I have more then one poller I can never be sure that all are running by monitoring the systemd service.

    Example: Harvest Systemd status is active

    ● harvest.service - Harvest
       Loaded: loaded (/etc/systemd/system/harvest.service; enabled; vendor preset: disabled)
       Active: active (running) since Tue 2021-05-25 09:56:15 CEST; 24min ago
      Process: 2512 ExecStart=/opt/harvest/bin/harvest restart --config /opt/harvest/harvest.yml (code=exited, status=0/SUCCESS)
     Main PID: 28539 (code=exited, status=0/SUCCESS)
        Tasks: 30
       Memory: 104.6M
       CGroup: /system.slice/harvest.service
               ├─2521 bin/poller --poller unix --loglevel 2 --config /opt/harvest/harvest.yml --daemon
               ├─2538 bin/poller --poller ontap1 --loglevel 2 --config /opt/harvest/harvest.yml --daemon
               └─2548 bin/poller --poller ontap2 --loglevel 2 --config /opt/harvest/harvest.yml --daemon
    

    I kill a single poller, service status remains active.

    kill 2548

    # ./bin/harvest status
    Datacenter            Poller                PID        PromPort        Status
    +++++++++++++++++++++ +++++++++++++++++++++ ++++++++++ +++++++++++++++ ++++++++++++++++++++
    local                   unix                2521                       running
    DC1                     ontap1              2538                       running
    DC2                     ontap2                                         not running
    +++++++++++++++++++++ +++++++++++++++++++++ ++++++++++ +++++++++++++++ ++++++++++++++++++++
    
    ● harvest.service - Harvest
       Loaded: loaded (/etc/systemd/system/harvest.service; enabled; vendor preset: disabled)
       Active: active (running) since Tue 2021-05-25 09:56:15 CEST; 25min ago
      Process: 2512 ExecStart=/opt/harvest/bin/harvest restart --config /opt/harvest/harvest.yml (code=exited, status=0/SUCCESS)
     Main PID: 28539 (code=exited, status=0/SUCCESS)
        Tasks: 20
       Memory: 63.2M
       CGroup: /system.slice/harvest.service
               ├─2521 bin/poller --poller unix --loglevel 2 --config /opt/harvest/harvest.yml --daemon
               └─2538 bin/poller --poller ontap1 --loglevel 2 --config /opt/harvest/harvest.yml --daemon
    

    Describe the solution you'd like Systemd harvest service should monitor the status of all pollers and should switch to "failed" if any poller is down. Alternatively create single service instances, one for each poller.

    Describe alternatives you've considered Monitoring the service and getting the status of every poller is essential for production usage.

    feature priority/P2 status/open 
    opened by mamoep 14
  • feature request: net-port-ifgrp-get-iter

    feature request: net-port-ifgrp-get-iter

    we are trying to get ifgroup info with custom zapi template. the only api i can find is "net-port-ifgrp-get". "show attrs" works. but "show data" gives the following error. seems net-port-ifgrp-get-iter is needed but not available.

    harvest zapi -p flc1-sys-ash0-cloudsys show data --api net-port-ifgrp-get connected to flc1-sys-ash0-cloudsys (NetApp Release 9.8P6: Tue Aug 03 16:21:11 UTC 2021) 2021/11/24 09:25:52 api request rejected => Missing input: ifgrp-name; Missing input: node

    if we specify parameters while using zapi directly, it can get a particular ifgrp on a node. but parameters are not supported in templates. we'd like to collect metrics[net-ifgrp-info] for all ifgrps on a filer.

    Thank you very much!

    James

    feature priority/P3 status/done 
    opened by pilot7777 13
  • Harvest service doesn't work as expected

    Harvest service doesn't work as expected

    Describe the bug A clear and concise description of what the bug is.

    When I started the service of harvest, it only runs two poller from 11 that we have defined in the harvest.yml. If I run the harvest process with the next command: **/opt/harvest/bin/harvest start** or **/opt/harvest/bin/harvest restart all the pollers** run correctly. It happens the same with the different context defining for the service that is to say: start, restart, status and stop.

    I attached an image of what we see in the beginning of the service

    image

    Environment Provide accurate information about the environment to help us reproduce the issue.

    • Harvest version: harvest version 21.05.1-1 (commit 2211c00) (build date 2021-05-21T01:28:12+0530) linux/amd64
    • Command line arguments used: [e.g. bin/harvest start --config=foo.yml --collectors Zapi]
    • OS: RHEL 8.2
    • Install method: yum
    • ONTAP Version: 9.5 and 9.7
    • Other:

    To Reproduce Running the service systemctl start harvest.service

    Expected behavior It should run a process for every poller in my harvest.yml.

    Actual behavior It only runs two pollers, sometimes none of them.

    Possible solution, workaround, fix Starting the gathering using the executable: "/opt/harvest/bin/harvest" instead of the service

    opened by faguayot 13
  • Poller is exiting with error no valid version tuple found"">

    Poller is exiting with error "connection error => no valid version tuple found"

    Describe the bug For few 7mode systems, harvest pollers are exiting with below errors.

    {"level":"warn","Poller":"netapp1","caller":"command-line-arguments/poller.go:530","time":"2021-08-02T21:57:30-07:00","message":"init collector-object (ZapiPerf:SystemNode): connection error => no valid version tuple found"} {"level":"warn","Poller":"netapp1","caller":"command-line-arguments/poller.go:532","time":"2021-08-02T21:57:30-07:00","message":"aborting collector (ZapiPerf)"} {"level":"warn","Poller":"netapp1","caller":"command-line-arguments/poller.go:255","time":"2021-08-02T21:57:30-07:00","message":"no collectors initialized, stopping"} {"level":"info","Poller":"netapp1","caller":"command-line-arguments/poller.go:419","time":"2021-08-02T21:57:30-07:00","message":"cleaning up and stopping [pid=14953]"}

    Environment Provide accurate information about the environment to help us reproduce the issue.

    • Harvest version: harvest version 21.05.3-2 (commit b482aff) (build date 2021-06-28T21:08:05+0530) linux/amd64
    • Command line arguments used: harvest start netapp1
    • OS: CentOS Linux release 7.9.2009 (Core)
    • Install method: rpm
    • ONTAP Version: 8.2P4 7mode
    • Other:

    To Reproduce Not sure how to reproduce

    Expected behavior Expect pollers to collect the metrics

    Actual behavior Poller exited with error

    Possible solution, workaround, fix NA

    Additional context NA

    bug status/done 
    opened by madhusudhanarya 12
  • Can't start the exporter prometheus

    Can't start the exporter prometheus

    Describe the bug We can't start the recollecion of data with prometheus for the new harvest releases 21.05.2 and the pre-release 21.05.3. In the previous version, we were running the collector without errors. Environment

    • Harvest version:harvest version 21.05.2-1 (commit ce091de) (build date 2021-06-14T20:31:09+0530) linux/amd64 or harvest version 21.05.3-1 (commit 63f0b11) (build date 2021-06-23T22:11:00+0530) linux/amd64

    • Command line arguments used: bin/harvest start --config two.yml

    • OS: RHEL 8.2

    • Install method: yum

    • ONTAP Version: 9.5 and 9.7

    To Reproduce Here the output of the command running in the foreground mode, here you can see the behaviour for the three different versions. In the 21.05.1 we were running correctly and in the other two not.

    ====================================
    HARVESST 21.05.1
    ====================================
    

    [[email protected] harvest]# bin/harvest -v harvest version 21.05.1-1 (commit 2211c00) (build date 2021-05-21T01:28:12+0530) linux/amd64 [[email protected] harvest]# bin/harvest start --config two.yml --foreground set debug mode ON (starting poller in foreground otherwise is unsafe) starting in foreground, enter CTRL+C or close terminal to stop poller 2021/06/24 01:18:42 (info ) : options config: two.yml 2021/06/24 01:18:42 (info ) (poller) (sces1p1_01): started in foreground [pid=1792] 2021/06/24 01:18:45 (info ) (poller) (sces1p1_01): poller start-up complete 2021/06/24 01:18:45 (info ) (poller) (sces1p1_01): updated status, up collectors: 41 (of 41), up exporters: 1 (of 1) 2021/06/24 01:18:45 (info ) (collector) (Zapi:SnapMirror): no [SnapMirror] instances on system, entering standby mode 2021/06/24 01:18:45 (info ) (collector) (Zapi:SnapMirror): no [SnapMirror] instances on system, entering standby mode 2021/06/24 01:18:45 (info ) (collector) (ZapiPerf:Path): no [Path] instances on system, entering standby mode 2021/06/24 01:18:45 (info ) (collector) (ZapiPerf:Path): recovered from standby mode, back to normal schedule 2021/06/24 01:18:45 (warning) (collector) (ZapiPerf:Path): lagging behind schedule 60.63µs 2021/06/24 01:18:46 (info ) (collector) (Zapi:Lun): no [Lun] instances on system, entering standby mode 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:WAFLSizer): no [WAFLSizer] instances on system, entering standby mode 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:WAFLSizer): recovered from standby mode, back to normal schedule 2021/06/24 01:18:46 (warning) (collector) (ZapiPerf:WAFLSizer): lagging behind schedule 61.529µs ^X2021/06/24 01:18:46 (error ) (collector) (ZapiPerf:FCVI): instance request: api request rejected => Counter collection is disabled 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:FCVI): no [FCVI] instances on system, entering standby mode 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:FCVI): recovered from standby mode, back to normal schedule 2021/06/24 01:18:46 (warning) (collector) (ZapiPerf:FCVI): lagging behind schedule 74.465µs 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:WAFLCompBin): no [WAFLCompBin] instances on system, entering standby mode 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:WAFLCompBin): recovered from standby mode, back to normal schedule 2021/06/24 01:18:46 (warning) (collector) (ZapiPerf:WAFLCompBin): lagging behind schedule 75.41µs 2021/06/24 01:18:46 (info ) (collector) (Zapi:Lun): no [Lun] instances on system, entering standby mode 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:WAFLAggr): no [WAFLAggr] instances on system, entering standby mode 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:WAFLAggr): recovered from standby mode, back to normal schedule 2021/06/24 01:18:46 (warning) (collector) (ZapiPerf:WAFLAggr): lagging behind schedule 56.449µs 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:CIFSvserver): no [CIFSvserver] instances on system, entering standby mode 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:CIFSvserver): recovered from standby mode, back to normal schedule 2021/06/24 01:18:46 (warning) (collector) (ZapiPerf:CIFSvserver): lagging behind schedule 185.021µs 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:ObjectStoreClient): no [ObjectStoreClient] instances on system, entering standby mode 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:ObjectStoreClient): recovered from standby mode, back to normal schedule 2021/06/24 01:18:46 (warning) (collector) (ZapiPerf:ObjectStoreClient): lagging behind schedule 59.669µs 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:ISCSI): no [ISCSI] instances on system, entering standby mode 2021/06/24 01:18:46 (info ) (collector) (ZapiPerf:ISCSI): recovered from standby mode, back to normal schedule 2021/06/24 01:18:46 (warning) (collector) (ZapiPerf:ISCSI): lagging behind schedule 64.995µs 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:NFSv3Node): no [NFSv3Node] instances on system, entering standby mode 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:NFSv3Node): recovered from standby mode, back to normal schedule 2021/06/24 01:18:47 (warning) (collector) (ZapiPerf:NFSv3Node): lagging behind schedule 354.255µs 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:NFSv4Node): no [NFSv4Node] instances on system, entering standby mode 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:NFSv4Node): recovered from standby mode, back to normal schedule 2021/06/24 01:18:47 (warning) (collector) (ZapiPerf:NFSv4Node): lagging behind schedule 46.863µs 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:ExtCacheObj): no [ExtCacheObj] instances on system, entering standby mode 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:ExtCacheObj): recovered from standby mode, back to normal schedule 2021/06/24 01:18:47 (warning) (collector) (ZapiPerf:ExtCacheObj): lagging behind schedule 188.952µs 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:NFSv3): no [NFSv3] instances on system, entering standby mode 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:NFSv3): recovered from standby mode, back to normal schedule 2021/06/24 01:18:47 (warning) (collector) (ZapiPerf:NFSv3): lagging behind schedule 73.488µs 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:CIFSNode): no [CIFSNode] instances on system, entering standby mode 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:CIFSNode): recovered from standby mode, back to normal schedule 2021/06/24 01:18:47 (warning) (collector) (ZapiPerf:CIFSNode): lagging behind schedule 57.672µs 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:NFSv41Node): no [NFSv41Node] instances on system, entering standby mode 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:NFSv41Node): recovered from standby mode, back to normal schedule 2021/06/24 01:18:47 (warning) (collector) (ZapiPerf:NFSv41Node): lagging behind schedule 62.548µs 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:NFSv4): no [NFSv4] instances on system, entering standby mode 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:NFSv4): recovered from standby mode, back to normal schedule 2021/06/24 01:18:47 (warning) (collector) (ZapiPerf:NFSv4): lagging behind schedule 64.687µs 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:NFSv41): no [NFSv41] instances on system, entering standby mode 2021/06/24 01:18:47 (info ) (collector) (ZapiPerf:NFSv41): recovered from standby mode, back to normal schedule 2021/06/24 01:18:47 (warning) (collector) (ZapiPerf:NFSv41): lagging behind schedule 82.519µs

     ====================================
     HARVESST 21.05.2
     ====================================
    

    [[email protected] harvest]# bin/harvest -v harvest version 21.05.2-1 (commit ce091de) (build date 2021-06-14T20:31:09+0530) linux/amd64 [[email protected] harvest]# bin/harvest start --config two.yml --foreground set debug mode ON (starting poller in foreground otherwise is unsafe) starting in foreground, enter CTRL+C or close terminal to stop poller 1:20AM INF command-line-arguments/poller.go:159 > log level used: info Poller=sces1p1_01 1:20AM INF command-line-arguments/poller.go:160 > options config: two.yml Poller=sces1p1_01 1:20AM INF command-line-arguments/poller.go:191 > started in foreground [pid=1895] Poller=sces1p1_01 1:20AM INF command-line-arguments/poller.go:293 > poller start-up complete Poller=sces1p1_01 1:20AM INF command-line-arguments/poller.go:417 > updated status, up collectors: 41 (of 41), up exporters: 1 (of 1) Poller=sces1p1_01 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [SnapMirror] instances on system, entering standby mode Poller=sces1p1_01 collector=Zapi:SnapMirror 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [Path] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:Path 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:Path 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 336.887µs Poller=sces1p1_01 collector=ZapiPerf:Path 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [SnapMirror] instances on system, entering standby mode Poller=sces1p1_01 collector=Zapi:SnapMirror 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [WAFLSizer] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:WAFLSizer 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:WAFLSizer 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 262.112µs Poller=sces1p1_01 collector=ZapiPerf:WAFLSizer 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [WAFLCompBin] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:WAFLCompBin 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:WAFLCompBin 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 81.755µs Poller=sces1p1_01 collector=ZapiPerf:WAFLCompBin 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [Lun] instances on system, entering standby mode Poller=sces1p1_01 collector=Zapi:Lun 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [ObjectStoreClient] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:ObjectStoreClient 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:ObjectStoreClient 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 171.365µs Poller=sces1p1_01 collector=ZapiPerf:ObjectStoreClient 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [ISCSI] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:ISCSI 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:ISCSI 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 112.106µs Poller=sces1p1_01 collector=ZapiPerf:ISCSI 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [CIFSvserver] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:CIFSvserver 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:CIFSvserver 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 275.845µs Poller=sces1p1_01 collector=ZapiPerf:CIFSvserver 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [CIFSNode] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:CIFSNode 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:CIFSNode 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 114.169µs Poller=sces1p1_01 collector=ZapiPerf:CIFSNode 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [NFSv3] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:NFSv3 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:NFSv3 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 92.334µs Poller=sces1p1_01 collector=ZapiPerf:NFSv3 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [WAFLAggr] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:WAFLAggr 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:WAFLAggr 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 85.435µs Poller=sces1p1_01 collector=ZapiPerf:WAFLAggr 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [Lun] instances on system, entering standby mode Poller=sces1p1_01 collector=Zapi:Lun 1:20AM ERR goharvest2/cmd/collectors/zapiperf/zapiperf.go:1163 > instance request error="api request rejected => Counter collection is disabled" Poller=sces1p1_01 collector=ZapiPerf:FCVI stack=[{"func":"New","line":"35","source":"errors.go"},{"func":"(*Client).invoke","line":"402","source":"client.go"},{"func":"(*Client).InvokeBatchWithTimers","line":"280","source":"client.go"},{"func":"(*Client).InvokeBatchRequest","line":"253","source":"client.go"},{"func":"(*ZapiPerf).PollInstance","line":"1162","source":"zapiperf.go"},{"func":"(*task).Run","line":"60","source":"schedule.go"},{"func":"(*AbstractCollector).Start","line":"270","source":"collector.go"},{"func":"goexit","line":"1371","source":"asm_amd64.s"}] 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [FCVI] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:FCVI 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:FCVI 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 95.872µs Poller=sces1p1_01 collector=ZapiPerf:FCVI 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [NFSv4] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:NFSv4 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:NFSv4 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 139.563µs Poller=sces1p1_01 collector=ZapiPerf:NFSv4 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [NFSv3Node] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:NFSv3Node 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:NFSv3Node 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 130.995µs Poller=sces1p1_01 collector=ZapiPerf:NFSv3Node 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [NFSv41Node] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:NFSv41Node 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:NFSv41Node 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 121.992µs Poller=sces1p1_01 collector=ZapiPerf:NFSv41Node 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [NFSv4Node] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:NFSv4Node 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:NFSv4Node 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 138.336µs Poller=sces1p1_01 collector=ZapiPerf:NFSv4Node 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [ExtCacheObj] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:ExtCacheObj 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:ExtCacheObj 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 144.945µs Poller=sces1p1_01 collector=ZapiPerf:ExtCacheObj 1:20AM INF goharvest2/cmd/poller/collector/collector.go:296 > no [NFSv41] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:NFSv41 1:20AM INF goharvest2/cmd/poller/collector/collector.go:318 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:NFSv41 1:20AM WRN goharvest2/cmd/poller/collector/collector.go:387 > lagging behind schedule 96.71µs Poller=sces1p1_01 collector=ZapiPerf:NFSv41

    ====================================
    HARVESST 21.05.3
    ====================================
    

    [[email protected] harvest]# bin/harvest -v harvest version 21.05.3-1 (commit 63f0b11) (build date 2021-06-23T22:11:00+0530) linux/amd64 [[email protected] harvest]# bin/harvest start --config two.yml --foreground set debug mode ON (starting poller in foreground otherwise is unsafe) configuration error => Poller does not exist sces1p1_01 starting in foreground, enter CTRL+C or close terminal to stop poller 1:21AM INF command-line-arguments/poller.go:154 > log level used: info Poller=sces1p1_01 1:21AM INF command-line-arguments/poller.go:155 > options config: two.yml Poller=sces1p1_01 1:21AM INF command-line-arguments/poller.go:180 > started in foreground [pid=2017] Poller=sces1p1_01 1:21AM INF command-line-arguments/poller.go:282 > poller start-up complete Poller=sces1p1_01 1:21AM INF command-line-arguments/poller.go:406 > updated status, up collectors: 41 (of 41), up exporters: 1 (of 1) Poller=sces1p1_01 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [SnapMirror] instances on system, entering standby mode Poller=sces1p1_01 collector=Zapi:SnapMirror 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [SnapMirror] instances on system, entering standby mode Poller=sces1p1_01 collector=Zapi:SnapMirror 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [Path] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:Path 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:Path 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 99.318µs Poller=sces1p1_01 collector=ZapiPerf:Path 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [WAFLSizer] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:WAFLSizer 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:WAFLSizer 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 134.251µs Poller=sces1p1_01 collector=ZapiPerf:WAFLSizer 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [WAFLCompBin] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:WAFLCompBin 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:WAFLCompBin 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 118.276µs Poller=sces1p1_01 collector=ZapiPerf:WAFLCompBin 1:21AM ERR goharvest2/cmd/collectors/zapiperf/zapiperf.go:1170 > instance request error="api request rejected => Counter collection is disabled" Poller=sces1p1_01 collector=ZapiPerf:FCVI stack=[{"func":"New","line":"35","source":"errors.go"},{"func":"(*Client).invoke","line":"403","source":"client.go"},{"func":"(*Client).InvokeBatchWithTimers","line":"281","source":"client.go"},{"func":"(*Client).InvokeBatchRequest","line":"254","source":"client.go"},{"func":"(*ZapiPerf).PollInstance","line":"1169","source":"zapiperf.go"},{"func":"(*task).Run","line":"60","source":"schedule.go"},{"func":"(*AbstractCollector).Start","line":"269","source":"collector.go"},{"func":"goexit","line":"1371","source":"asm_amd64.s"}] 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [FCVI] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:FCVI 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:FCVI 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 75.559µs Poller=sces1p1_01 collector=ZapiPerf:FCVI 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [WAFLAggr] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:WAFLAggr 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:WAFLAggr 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 81.203µs Poller=sces1p1_01 collector=ZapiPerf:WAFLAggr 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [Lun] instances on system, entering standby mode Poller=sces1p1_01 collector=Zapi:Lun 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [ObjectStoreClient] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:ObjectStoreClient 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:ObjectStoreClient 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 133.832µs Poller=sces1p1_01 collector=ZapiPerf:ObjectStoreClient 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [CIFSvserver] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:CIFSvserver 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:CIFSvserver 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 5.773295ms Poller=sces1p1_01 collector=ZapiPerf:CIFSvserver 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [ISCSI] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:ISCSI 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:ISCSI 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 83.647µs Poller=sces1p1_01 collector=ZapiPerf:ISCSI 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [ExtCacheObj] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:ExtCacheObj 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:ExtCacheObj 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 115.696µs Poller=sces1p1_01 collector=ZapiPerf:ExtCacheObj 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [CIFSNode] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:CIFSNode 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:CIFSNode 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 156.771µs Poller=sces1p1_01 collector=ZapiPerf:CIFSNode 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [Lun] instances on system, entering standby mode Poller=sces1p1_01 collector=Zapi:Lun 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [NFSv4Node] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:NFSv4Node 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:NFSv4Node 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 80.038µs Poller=sces1p1_01 collector=ZapiPerf:NFSv4Node 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [NFSv3Node] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:NFSv3Node 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:NFSv3Node 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 686.415µs Poller=sces1p1_01 collector=ZapiPerf:NFSv3Node 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [NFSv41Node] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:NFSv41Node 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:NFSv41Node 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 190.802µs Poller=sces1p1_01 collector=ZapiPerf:NFSv41Node 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [NFSv3] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:NFSv3 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:NFSv3 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 327.095µs Poller=sces1p1_01 collector=ZapiPerf:NFSv3 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [NFSv4] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:NFSv4 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:NFSv4 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 81.35µs Poller=sces1p1_01 collector=ZapiPerf:NFSv4 1:21AM INF goharvest2/cmd/poller/collector/collector.go:295 > no [NFSv41] instances on system, entering standby mode Poller=sces1p1_01 collector=ZapiPerf:NFSv41 1:21AM INF goharvest2/cmd/poller/collector/collector.go:317 > recovered from standby mode, back to normal schedule Poller=sces1p1_01 collector=ZapiPerf:NFSv41 1:21AM WRN goharvest2/cmd/poller/collector/collector.go:386 > lagging behind schedule 120.266µs Poller=sces1p1_01 collector=ZapiPerf:NFSv41

    opened by faguayot 12
  • Tagging of Volume

    Tagging of Volume

    Customer would like to have capability to tag volumes for business units, workloads, projects, cost center etc etc.

    In customer environment volume names are using artificial, random names during the provisioning process. For proper identification they would like tags for project, cost-center etc

    Q: Can you please advise us on how to add custom tags to time series for volumes. Like BusinessUnit, Project, UsageModel etc…
    A: Prometheus supports labels (which can be treated like tags). A counter can have many labels. Typically we assign labels to counters like: • datacenter • cluster • vserver • volume • …

    Labels get assigned during ingest into the database. You cannot (like with all TSDBs) add tags or labels afterwards. If you want to add custom labels to volumes as outlined below, there are actually three options to do that:

    1. you use the comment filed in ONTAP itself – you need to create a plugin in Harvest to decode the comment field and create labels for the volume. We would recommend to create a new counter per volume which holds all these labels - this is more efficient and can be used for filtering in Grafana
    2. You load this new volume counter manually into Prometheus using the file exporter capability of Prometheus node exporter
    3. You create a new collector which generates these counters

    It’s more the question who and when do you want to assign these tags to the volume

    For option 1: Can the netapp team create that plugin… I am sure this is something that all your customer base will find useful.

    Q: Sorry if this is a repeated question, but can you provide 1-2 examples of the comment section and the labels you want to extract? A: How about the following

    Comment= “ Label1:

    Each label:label data separated by spaces.. You can interpret as many labels or set a limit of 10 labels.

    Label data and label names cannot have any special characters other than “_ or – “ and alphanumeric characters only

    We can start with volume management. But if easy to implement for other objects we should. But volume is priority

    Q: If the Zapi object does provide the comment as a metric, then it's easy and probably no need to write a new plugin for another object. One last question that might sound a bit strange, but makes a big difference how the plugin would be implemented: how are you planning to use these labels? Do you want to be able to filter/select the metrics based on these labels? Or is it sufficient for you to get a table showing these labels (e.g. which volumes have which owners, projects, etc.).

    A: We expect to filter based on these metrics and do various operation. Also we plan to implement alerting and monitoring also based on these filters.

    Example: If utilization >90 for businessunit:xyz then send email or raise alert etc…

    feature priority/P1 status/done 
    opened by demalik 12
  • Logs report Error for No Instances Found, shouldn't this be info and not error?

    Logs report Error for No Instances Found, shouldn't this be info and not error?

    Is your feature request related to a problem? Please describe. Not a problem, but it interrupts with error debugging.

    Logs show Errors when it does not find quota instances on the clusters without any quota. So when I analyze issues, I have to double check on cluster to see if it is really an error or tree quota does not exist. This adds one more layer of checks to make sure it is not really an error, its just that particular collection does not find the instance.

    {"level":"error","Poller":"POLLER","collector":"Zapi:Qtree","stack":[{"func":"New","line":"36","source":"errors.go"},{"func":"(*Qtree).Run","line":"160","source":"qtree.go"},{"func":"(*AbstractCollector).Start","line":"376","source":"collector.go"},{"func":"goexit","line":"1571","source":"asm_amd64.s"}],"error":"no instances => no quota instances found","caller":"goharvest2/cmd/poller/collector/collector.go:377","time":"2022-07-29T10:41:13-07:00","message":"plugin [Qtree]: "}

    Describe the solution you'd like

    Is it possible to log this type of scenarios as information and not Errors? That way it is easy to ignore these from logs processing and analysis.

    Describe alternatives you've considered I just do regex to remove some of this, but it still requires me to check on cluster first before excluding.

    feature 
    opened by jmg011 2
  • Dashboard to identify busy volumes in an aggregate NABOX

    Dashboard to identify busy volumes in an aggregate NABOX

    Discussed in https://github.com/NetApp/harvest/discussions/1137

    Originally posted by swordfish291 July 12, 2022 Hello Team,

    I am using latest version of NaBox and I was looking for a dashboard that can provide IOPS/Throughput/Latency details of all the volumes in an aggregate. I am looking to identify busy volumes in an aggregate causing high disk utilization.

    Thank you Abdul

    feature customer 
    opened by cgrinds 0
  • Collector panicked

    Collector panicked

    Describe the bug

    A collector panicked:

    1:28PM ERR goharvest2/cmd/poller/collector/collector.go:240 > Collector panicked error="goroutine error..." Poller=REDACTED collector=ZapiPerf:Volume stack=[{"func":"New","line":"35","source":"errors.go"},{"func":"(*AbstractCollector).Start.func1","line":"240","source":"collector.go"},{"func":"gopanic","line":"965","source":"panic.go"},{"func":"goPanicIndex","line":"88","source":"panic.go"},{"func":"(*MetricFloat64).SetValueFloat64","line":"97","source":"metric_float64.go"},{"func":"(*ZapiPerf).PollData","line":"326","source":"zapiperf.go"},{"func":"(*task).Run","line":"60","source":"schedule.go"},{"func":"(*AbstractCollector).Start","line":"269","source":"collector.go"},{"func":"goexit","line":"1371","source":"asm_amd64.s"}]
    

    I'm running two instances of this collector in parallel running against the same cluster, both panicked simultaneously - the collector Pod continued running but stopped exporting some Prometheus metrics such as volume_total_ops.

    Seems pretty similar to https://github.com/NetApp/harvest/issues/174#issuecomment-861495148.

    Environment

    • Harvest version: harvest:21.05.3-2
    • Command line arguments used: bin/poller --poller REDACTED --promPort 12991 --config /opt/harvest/harvest.yaml
    • OS: Docker 19.03.1-CE
    • Install method: docker
    status/needs-triage 
    opened by bilbof 3
  • Flexgroup Read Latency

    Flexgroup Read Latency

    Describe the bug Flexgroup read latency is incorrect.

    Environment ONTAP 9.10.1P3 , NABOX 3.1 (install method: OVA image)

    To Reproduce Create a FlexGroup on Netapp Excute a workload (read/write operations) Open NABOX to watch latencies Open Dashboard: Harvest - cDOT / NetApp Detail: Volume - Details Open the row : Volume WAFL Layer Drilldown Look the panel: Read Latency Metric: topk($TopResources, volume_read_latency{datacenter="$Datacenter",cluster="$Cluster",svm=~"$SVM",volume=~"$Volume"})

    Expected behavior Latencies similar to those obtained with the ONTAP CLI command "qos statistics volume latency show", below 5ms.

    Actual behavior NABOX displays read latencies above 15ms.

    Possible solution, workaround, fix Looks like some metrics (volume_read_latency, volume_avg_latency) are incorrect for flexgroup volumes. and its constituents. It works fine for flexvols.

    Additional context Here is the output latencies from qos statistics volume latency show:

    Workload ID Latency Network Cluster Data Disk QoS Max QoS Min NVRAM Cloud FlexCache SM Sync VA AVSCAN


    -total- - 515.00us 65.00us 137.00us 302.00us 0ms 0ms 0ms 11.00us 0ms 0ms 0ms 0ms 0ms -total- - 512.00us 68.00us 132.00us 301.00us 0ms 0ms 0ms 11.00us 0ms 0ms 0ms 0ms 0ms -total- - 499.00us 61.00us 131.00us 296.00us 0ms 0ms 0ms 11.00us 0ms 0ms 0ms 0ms 0ms -total- - 521.00us 62.00us 132.00us 282.00us 34.00us 0ms 0ms 11.00us 0ms 0ms 0ms 0ms 0ms -total- - 501.00us 63.00us 130.00us 296.00us 0ms 0ms 0ms 12.00us 0ms 0ms 0ms 0ms 0ms -total- - 474.00us 61.00us 131.00us 271.00us 0ms 0ms 0ms 11.00us 0ms 0ms 0ms 0ms 0ms -total- - 492.00us 64.00us 132.00us 285.00us 0ms 0ms 0ms 11.00us 0ms 0ms 0ms 0ms 0ms -total- - 506.00us 72.00us 134.00us 289.00us 0ms 0ms 0ms 11.00us 0ms 0ms 0ms 0ms 0ms -total- - 465.00us 60.00us 127.00us 267.00us 0ms 0ms 0ms 11.00us 0ms 0ms 0ms 0ms 0ms -total- - 491.00us 65.00us 129.00us 286.00us 0ms 0ms 0ms 11.00us 0ms 0ms 0ms 0ms 0ms -total- - 474.00us 61.00us 131.00us 271.00us 0ms 0ms 0ms 11.00us 0ms 0ms 0ms 0ms 0ms -total- - 498.00us 61.00us 133.00us 293.00us 0ms 0ms 0ms 11.00us 0ms 0ms 0ms 0ms 0ms -total- - 514.00us 63.00us 132.00us 308.00us 0ms 0ms 0ms 11.00us 0ms 0ms 0ms 0ms 0ms

    Thanks

    customer 
    opened by josepaulog 11
Releases(nightly)
  • nightly(Aug 1, 2022)

  • v22.05.0(May 11, 2022)

    Releases

    22.05.0 / 2022-05-11

    :rocket: Highlights of this major release include:

    • Early access to ONTAP REST perf collector from ONTAP 9.11.1GA+

    • :hourglass: New Container Registry - Several of you have mentioned that you are being rate-limited when pulling Harvest Docker images from DockerHub. To alleviate this problem, we're publishing Harvest images to NetApp's container registry (cr.netapp.io). Going forward, we'll publish images to both DockerHub and cr.netapp.io. More information in the FAQ. No action is required unless you want to switch from DockerHub to cr.netapp.io. If so, the FAQ has you covered.

    • Five new dashboards added in this release

      • Power dashboard
      • Compliance dashboard
      • Security dashboard
      • Qtree dashboard
      • NFSv4 Store Pool dashboard (disabled by default)
    • New value_to_num_regex plugin allows you to map all matching expressions to 1 and non-matching ones to 0.

    • Harvest pollers can optionally read credentials from a mounted volume or file. This enables Hashicorp Vault support and works especially well with Vault agent

    • bin/grafana import provides a --multi flag that rewrites dashboards to include multi-select dropdowns for each variable at the top of the dashboard

    • The conf/rest collector templates received a lot of attentions this release. All known gaps between the ZAPI and REST collector have been filled and there is full parity between the two from ONTAP 9.11+. :metal:

    • 24 bug fixes, 48 feature, and 5 documentation commits this release

    IMPORTANT :bangbang: After upgrade, don't forget to re-import your dashboards so you get all the new enhancements and fixes. You can import via bin/harvest/grafana import cli or from the Grafana UI.

    IMPORTANT The conf/zapiperf/cdot/9.8.0/object_store_client_op.yaml ZapiPerf template is being deprecated in this release and will be removed in the next release of Harvest. No dashboards use the counters defined in this template and all counters are being deprecated by ONTAP. If you are using these counters, please create your own copy of the template.

    Known Issues

    IMPORTANT 7-mode filers that are not on the latest release of ONTAP may experience TLS connection issues with errors like tls: server selected unsupported protocol version 301 This is caused by a change in Go 1.18. The default for TLS client connections was changed to TLS 1.2 in Go 1.18. Please upgrade your 7-mode filers (recommended) or set tls_min_version: tls10 in your harvest.yml poller section. See #1007 for more details.

    The Unix collector is unable to monitor pollers running in containers. See #249 for details.

    Enhancements

    • Harvest should include a Power dashboard that shows power consumed, temperatures and fan speeds at a node and shelf level #932 and #903

    • Harvest should include a Security dashboard that shows authentication methods and certificate expiration details for clusters, volume encryption and status of anti-ransomware for volumes and SVMs #935

    • Harvest should include a Compliance dashboard that shows compliance status of clusters and SVMs along with individual compliance attributes #935

    • SVM dashboard should show antivirus counters in the CIFS drill-down section #913 Thanks to @burkl for reporting

    • Cluster and Aggregate dashboards should show Storage Efficiency Ratio metrics #888 Thanks to @Falcon667 for reporting

    • :construction: This is another step in the ZAPI to REST road map. In earlier releases, we focused on config ZAPIs and in this release we've added early access to an ONTAP REST perf collector. :confetti_ball: The REST perf collector and thirty-nine templates included in this release, require ONTAP 9.11.1GA+ :astonished: These should be considered early access as we continue to improve them. If you try them out or have any feedback, let us know on Slack or GitHub #881

    • Harvest should collect NFS v4.2 counters which are new in ONTAP 9.11+ releases #572

    • Plugin logging should include object detail #986

    • Harvest dashboards should use Time series panels instead of Graph (old) panels #972. Thanks to @ybizeul for raising

    • New regex based plugin value_to_num_regex helps map labels to numeric values for Grafana dashboards.

    • Harvest status should run on systems without pgrep #937 Thanks to Dan Butler for reporting this on Slack

    • When using a credentials file and the poller is not found, also consult the defaults section of the harvest.yml file #936

    • Harvest should include an NFSv4 StorePool dashboard that shows NFSv4 store pool locks and allocation detail #921 Thanks to Rusty Brown for contributing this dashboard.

    • REST collector should report cpu-busytime for node #918 Thanks to @pilot7777 for reporting this on Slack

    • Harvest should include a Qtree dashboard that shows Qtree NFS/CIFS metrics #812 Thanks to @ev1963 for reporting

    • Harvest should support reading credentials from an external file or mounted volume #905

    • Grafana dashboards should have checkbox to show multiple objects in variable drop-down. See comment for details. #815 #939 Thanks to @manuelbock, @bcase303 for reporting

    • Harvest should include Prometheus port (promport) to metadata metric #878

    • Harvest should use NetApp's container registry for Docker images #874

    • Increase ZAPI client timeout for default and volume object #1005

    • REST collector should support retrieving a subset of objects via template filtering support #950

    • Harvest should support minimum TLS version config #1007 Thanks to @jmg011 for reporting and verifying this

    Fixes

    • SVM Latency numbers differ significantly on Harvest 1.6 vs Harvest 2.0 #1003 See discussion as well. Thanks to @jmg011 for reporting

    • Harvest should include regex patterns to ignore transient volumes related to backup #929. Not enabled by default, see conf/zapi/cdot/9.8.0/volume.yaml for details. Thanks to @ybizeul for reporting

    • Exclude OS aggregates from capacity used graph #327 Thanks to @matejzero for raising

    • Few panels need to have instant property in Data protection dashboard #945

    • CPU overload when there are several thousands of quotas #733 Thanks to @Flo-Fly for reporting

    • Include 7-mode CLI role commands for Harvest user #891 Thanks to @ybizeul for reporting and providing the changes!

    • Zapi Collector fails to collect data if number of records on a poller is equal to batch size #870 Thanks to @unbreakabl3 on Slack for reporting

    • Wrong object name used in conf/zapi/cdot/9.8.0/snapshot.yaml #862 Thanks to @pilot7777 for reporting

    • Field access-time returned by snapshot-get-iter should be creation-time #861 Thanks to @pilot7777 for reporting

    • Harvest panics when trying to merge empty template #859 Thanks to @pilot7777 for raising

    Source code(tar.gz)
    Source code(zip)
    harvest-22.05.0-1.amd64.deb(58.35 MB)
    harvest-22.05.0-1.x86_64.rpm(48.38 MB)
    harvest-22.05.0-1_linux_amd64.tar.gz(63.01 MB)
  • v.22.02.0(Feb 15, 2022)

    Releases

    22.02.0 / 2022-02-15

    :boom: Highlights of this major release include:

    • Continued progress on the ONTAP REST config collector. Most of the template changes are in place and we're working on closing the gaps between ZAPI and REST. We've made lots of improvements to the REST collector and included 13 REST templates in this release. The REST collector should be considered early-access as we continue to improve it. If you try it out or have any feedback, let us know on Slack or GitHub. :book: You can find more information about when you should switch from ZAPI to REST, what versions of ONTAP are supported by Harvest's REST collector, and how to fill ONTAP gaps between REST and ZAPI documented here

    • Many of you asked for nightly builds. We have them. :confetti_ball: We're also working on publishing to multiple Docker registries since you've told us you're running into rate-limiting problems with DockerHub. We'll announce here and Slack when we have a solution in place.

    • Two new Data Protection dashboards

    • bin/grafana cli should not overwrite dashboard changes, making it simpler to import/export dashboards, and enabling round-tripping dashboards (import, export, re-import)

    • New include_contains plugin allows you to select a subset of objects. e.g. selecting only volumes with custom-defined ONTAP metadata

    • We've included more out-of-the-box Prometheus alerts. Keep sharing your most useful alerts!

    • 7mode workflows continue to be improved :heart: Harvest now collects Qtree and Quotas counters from 7mode filers (these are already collected in cDOT)

    • 28 bug fixes, 52 feature, and 11 documentation commits this release

    IMPORTANT Admin node certificate file location changed. Certificate files have been consolidated into the cert directory. If you created self-signed admin certs, you need to move the admin-cert.pem and admin-key.pem files into the cert directory.

    IMPORTANT In earlier versions of Harvest, the Qtree template exported the vserver metric. This counter was changed to svm to be consistent with other templates. If you are using the qtree vserver metric, you will need to update your queries to use svm instead.

    IMPORTANT :bangbang: After upgrade, don't forget to re-import your dashboards so you get all the new enhancements and fixes. You can import via bin/harvest/grafana import cli or from the Grafana UI.

    IMPORTANT The LabelAgent value_mapping plugin was deprecated in the 21.11 release and removed in 22.02. Use LabelAgent value_to_num instead. See docs for details.

    Known Issues

    The Unix collector is unable to monitor pollers running in containers. See #249 for details.

    Enhancements

    • Harvest should include a Data Protection dashboard that shows volumes protected by snapshots, which ones have exceeded their reserve copy, and which are unprotected #664

    • Harvest should include a Data Protection SnapMirror dashboard that shows which volumes are protected, how they're protected, their protection relationship, along with their health and lag durations.

    • Harvest should provide nightly builds to GitHub and DockerHub #713

    • Harvest bin/grafana cli should not overwrite dashboard changes, making it simpler to import/export dashboards, and enabling round-tripping dashboards (import, export, re-import) #831 Thanks to @luddite516 for reporting and @florianmulatz for iterating with us on a solution

    • Harvest should provide a include_contains label agent plugin for filtering #735 Thanks to @chadpruden for reporting

    • Improve Harvest's container compatibility with K8s via kompose. #655 See also and discussion

    • The ZAPI cli tool should include counter types when querying ZAPIs #663

    • Harvest should include a richer set of Prometheus alerts #254 Thanks @demalik for raising

    • Template plugins should run in the order they are defined and compose better. The output of one plugin can be fed into the input of the next one. #736 Thanks to @chadpruden for raising

    • Harvest should collect Antivirus counters when ONTAP offbox vscan is configured #346 Thanks to @burkl and @Falcon667 for reporting

    • Document how to run Harvest with containerd and Rancher

    • Qtree counters should be collected for 7-mode filers #766 Thanks to @jmg011 for raising this issue and iterating with us on a solution

    • Harvest admin node should work with pollers running in Docker compose #678

    • Document how to run Harvest with Podman. Several RHEL customers asked about this since Podman ships as the default container runtime on RHEL8+.

    • Harvest should include a Systemd service file for the HTTP service discovery admin node #656

    • Document how ZAPI collectors, templates, and exporting work together. Thanks @jmg011 and others for asking for this

    • Remove redundant dashboards (Network, Node, SVM, Volume) #703 Thanks to @mamoep for reporting this

    • Harvest generate docker command should support customer-supplied Prometheus and Grafana ports. #584

    • Harvest certificate authentication should work with self-signed subject alternative name (SAN) certificates. Improve documentation on how to use certificate authentication. Thanks to @edd1619 for raising this issue

    • Harvest's Prometheus exporter should optionally sort labels. Without sorting, VictoriaMetrics marks metrics stale. #756 Thanks to @mamoep for reporting and verifying

    • Harvest should optionally log to a file when running in the foreground. Handy for instantiated instances running on OSes that have poor support for jounalctl #813 and #810 Thanks to @mamoep for reporting and verifying this works in a nightly build

    • Harvest should collect workload concurrency #714

    • Harvest certificate directory should be included in a container's volume mounts #725

    • MetroCluster dashboard should show path object metrics #746

    • Harvest should collect namespace resources from ONTAP #749

    • Harvest should be more resilient to cluster connectivity issues #480

    • Harvest Grafana dashboard version string should match the Harvest release #631

    • REST collector improvements

      • Harvest REST collector should support ONTAP private cli endpoints #766

      • REST collector should support ZAPI-like object prefixing #786

      • REST collector should support computing new customer-defined metrics #780

      • REST collector should collect aggregate, qtree and quota counters #780

      • REST collector metrics should be reported in autosupport #841

      • REST collector should collect sensor counters #789

      • Collect network port interface information not available via ZAPI #691 Thanks to @pilot7777, @mamoep amd @wagneradrian92 for working on this with us

      • Publish REST collector document that highlights when you should switch from ZAPI to REST, what versions of ONTAP are supported by Harvest's REST collectors and how to fill ONTAP gaps between REST and ZAPI

      • REST collector should support Qutoa, Shelf, Snapmirror, and Volume plugins #799 and #811

    • Improve troubleshooting and documentation on validating certificates from macOS #723

    • Harvest should read its config information from the environment variable HARVEST_CONFIG when supplied. This env var has higher precedence than the --config command-line argument. #645

    Fixes

    • FlexGroup statistics should be aggregated across node and aggregates #706 Thanks to @wally007 for reporting

    • Network Details dashboard should use correct units and support variable sorting #673 Thanks to @mamoep for reporting and reviewing the fix

    • Harvest Systemd service should wait for network to start #707 Thanks to @mamoep for reporting and fixing

    • MetroCluster dashboard should use correct units and support variable sorting #685 Thanks to @mamoep and @chris4789 for reporting this

    • 7mode shelf plugin should handle cases where multiple channels have the same shelf id #692 Thanks to @pilot7777 for reporting this on Slack

    • Improve template YAML parsing when indentation varies #704 Thanks to @mamoep for reporting this.

    • Harvest should not include version information in its container name. #660. Thanks to @wally007 for raising this.

    • Ignore missing Qtrees and improve uniqueness check on 7mode filers #782 and #797. Thanks to @jmg011 for reporting

    • Qtree instance key order should be unified between 7mode and cDOT #807 Thanks to @jmg011 for reporting

    • Workload detail volume collection should not try to create duplicate counters #803 Thanks to @luddite516 for reporting

    • Harvest HTTP service discovery node should not attempt to publish Prometheus metrics to InfluxDB #684

    • Grafana import should save auth token to the config file referenced by HARVEST_CONFIG when that environnement variable exists #681

    • bin/zapi should print output #715

    • Snapmirror dashboard should show correct number of SVM-DR relationships, last transfer, and health status #728 Thanks to Gaël Cantarero on Slack for reporting

    • Ensure that properties defined in object templates override their parent properties #765

    • Increase time that metrics are retained in Prometheus exporter from 3 minutes to 5 minutes #778

    • Remove the misplaced SVM FCP Throughput panel from the iSCSI drilldown section of the SVM details dashboard #821 Thanks to @florianmulatz for reporting and fixing

    • When importing Grafana dashboards, remove the existing id and uid so Grafana treats the import as a create instead of an overwrite #825 Thanks to @luddite516 for reporting

    • Relax the Grafana version check constraint so version 8.4.0-beta1 is considered >=7.1 #828 Thanks to @ybizeul for reporting

    • bin/harvest status should report running for pollers exporting to InfluxDB, instead of reporting that they are not running #835

    • Pin the Grafana and Prometheus versions in the Docker compose workflow instead of pulling latest #822

    Source code(tar.gz)
    Source code(zip)
    harvest-22.02.0-4.amd64.deb(56.12 MB)
    harvest-22.02.0-4.x86_64.rpm(47.15 MB)
    harvest-22.02.0-4_linux_amd64.tar.gz(60.56 MB)
  • v21.11.1(Dec 10, 2021)

    Change Log

    Releases

    21.11.1 / 2021-12-10

    This release is the same as 21.11.0 with an FSx dashboard fix for #737. If you are not monitoring an FSx system the 21.11.0 release is the same, no need to upgrade. We reverted a node-labels check in those dashboards because Harvest does not collect node data from FSx systems.

    Highlights of this major release include:

    • Early access to ONTAP REST collector
    • Support for Prometheus HTTP service discovery
    • New MetroCluster dashboard
    • Qtree and Quotas collection
    • We heard your ask, and we made it happen. We've separated cDOT and 7mode dashboards so each can evolve independently
    • Label sets allow you to add additional key-value pairs to a poller's metrics #538 and expose those labels in your dashboards
    • Template merging was improved to keep your template changes separate from Harvest's
    • Harvest poller's are more deterministic about picking free ports
    • 37 bug fixes

    IMPORTANT The LabelAgent value_mapping plugin is being deprecated in this release and will be removed in the next release of Harvest. Use LabelAgent value_to_num instead. See docs for details.

    IMPORTANT After upgrade, don't forget to re-import all dashboards so you get new dashboard enhancements and fixes. You can re-import via bin/harvest/grafana cli or from the Grafana UI.

    IMPORTANT RPM and Debian packages will be deprecated in the future, replaced with Docker and native binaries. See #330 for details and tell us what you think. Several of you have already weighed-in. Thanks! If you haven't, please do.

    Known Issues

    The Unix collector is unable to monitor pollers running in containers. See #249 for details.

    Enhancements

    • :construction: ONTAP started moving their APIs from ZAPI to REST in ONTAP 9.6. Harvest adds an early access ONTAP REST collector in this release (config only). :confetti_ball: This is our first step among several as we prepare for the day that ZAPIs are turned off. The REST collector and seven templates are included in 21.11. These should be considered early access as we continue to improve them. If you try them out or have any feedback, let us know on Slack or GitHub. #402

    • Harvest should have a Prometheus HTTP service discovery end-point to make it easier to add/remove pollers #575

    • Harvest should include a MetroCluster dashboard #539 Thanks @darthVikes for reporting

    • Harvest should collect Qtree and Quota metrics #522 Thanks @jmg011 for reporting and validating this works in your environment

    • SVM dashboard: Make NFS version a variable. SVM variable should allow selecting all SVMs for a cluster wide view #454

    • Harvest should monitor ONTAP chassis sensors #384 Thanks to @hashi825 for raising this issue and reviewing the pull request

    • Harvest cluster dashboard should include All option in dropdown for clusters #630 thanks @TopSpeed for raising this on Slack

    • Harvest should collect volume sis status #519 Thanks to @jmg011 for raising

    • Separate cDOT and 7-mode dashboards allowing each to change independently #489 #501 #547

    • Improve collector and object template merging and documentation #493 #555 Thanks @hashi825 for reviewing and suggesting improvements

    • Harvest should support label sets, allowing you to add additional key-value pairs to a poller's metrics#538

    • bin/grafana import should create a matching label and rewrite queries to use chained variable when using label sets #550

    • Harvest poller's should reuse their previous Prometheus port when restarted and be more deterministic about picking free ports #596 #595 Thanks to @cordelster for reporting

    • Improve instantiated systemd template by specifying user/group, requires, and moving Unix pollers to the end of the list. #643 Thanks to @mamoep for reporting and providing the changes! :sparkles:

    • Harvest's Docker container should use local conf directory instead of copying into image. Makes upgrade and changing template files easier. #511

    • Improve Disk dashboard by showing total number of disks by node and aggregate #583

    • Harvest 7-mode dashboards should be provisioned when using Docker Compose workflow #544

    • When upgrading, bin/harvest grafana import should add dashboards to a release-named folder so earlier dashboards are not overwritten #616

    • client_timeout should be overridable in object template files #563

    • Increase ZAPI client timeouts for volume and workloads objects #617

    • Doctor: Ensure that all pollers export to unique Prometheus ports #597

    • Improve execution performance of Harvest management commands :rocket: bin/harvest start|stop|restart #600

    • Include eight cDOT dashboards that use InfluxDB datasource #466. Harvest does not support InfluxDB dashboards for 7-mode. Thanks to @SamyukthaM for working on these

    • Docs: Describe how Harvest converts template labels into Prometheus labels #585

    • Docs: Improve Matrix documentation to better align with code #485

    • Docs: Improve ARCHITECTURE.md #603

    Fixes

    • Poller should report metadata when running on BusyBox #529 Thanks to @charz for reporting issue and providing details

    • Space used % calculation was incorrect for Cluster and Aggregate dashboards #624 Thanks to @faguayot and @jorbour for reporting.

    • When ONTAP indicates a counter is deprecated, but a replacement is not provided, continue using the deprecated counter #498

    • Harvest dashboard panels must specify a Prometheus datasource to correctly handles cases were a non-Prometheus default datasource is defined in Grafana. #639 Thanks for reporting @MrObvious

    • Prometheus datasource was missing on five dashboards (Network and Disk) #566 Thanks to @survive-wustl for reporting

    • Document permissions that Harvest requires to monitor ONTAP with a read-only user #559 Thanks to @survive-wustl for reporting and working with us to chase this down. :thumbsup:

    • Metadata dashboard should show correct status for running/stopped pollers #567 Thanks to @cordelster for reporting

    • Harvest should serve a human-friendly :corn: overview page of metric types when hitting the Prometheus end-point #613 Thanks @cordelster for reporting

    • SnapMirror plugin should include source_node #608

    • Disk dashboard should use better labels in table details #578

    • SVM dashboard should show correct units and remove duplicate graph #454

    • FCP plugin should work with 7-mode clusters #464

    • Node values are missing from some 7-mode perf counters #467

    • Nic state is missing from several network related dashboards 486

    • Reduce log noise when templates are not found since this is often expected #606

    • Use diagnosis-config-get-iter to collect node status from 7-mode systems #499

    • Node status is missing from 7-mode #527

    • Improve 7-mode templates. Remove cluster from 7-mode. yamllint all templates #531

    • When saving Grafana auth token, make sure bin/grafana writes valid Yaml #544

    • Improve Yaml parsing when different levels of indention are used in harvest.yml. You should see fewer invalid indentation messages. :clap: #626

    • Unix poller should ignore /proc files that aren't readable #249

    Source code(tar.gz)
    Source code(zip)
    harvest-21.11.1-1.amd64.deb(54.40 MB)
    harvest-21.11.1-1.x86_64.rpm(45.42 MB)
    harvest-21.11.1-1_linux_amd64.tar.gz(58.75 MB)
  • v21.11.0(Nov 8, 2021)

    Change Log

    Releases

    21.11.0 / 2021-11-08

    Highlights of this major release include:

    • Early access to ONTAP REST collector
    • Support for Prometheus HTTP service discovery
    • New MetroCluster dashboard
    • Qtree and Quotas collection
    • We heard your ask, and we made it happen. We've separated cDOT and 7mode dashboards so each can evolve independently
    • Label sets allow you to add additional key-value pairs to a poller's metrics #538 and expose those labels in your dashboards
    • Template merging was improved to keep your template changes separate from Harvest's
    • Harvest poller's are more deterministic about picking free ports
    • 37 bug fixes

    IMPORTANT The LabelAgent value_mapping plugin is being deprecated in this release and will be removed in the next release of Harvest. Use LabelAgent value_to_num instead. See docs for details.

    IMPORTANT After upgrade, don't forget to re-import all dashboards so you get new dashboard enhancements and fixes. You can re-import via bin/harvest/grafana cli or from the Grafana UI.

    IMPORTANT RPM and Debian packages will be deprecated in the future, replaced with Docker and native binaries. See #330 for details and tell us what you think. Several of you have already weighed-in. Thanks! If you haven't, please do.

    Known Issues

    The Unix collector is unable to monitor pollers running in containers. See #249 for details.

    Enhancements

    • :construction: ONTAP started moving their APIs from ZAPI to REST in ONTAP 9.6. Harvest adds an early access ONTAP REST collector in this release (config only). :confetti_ball: This is our first step among several as we prepare for the day that ZAPIs are turned off. The REST collector and seven templates are included in 21.11. These should be considered early access as we continue to improve them. If you try them out or have any feedback, let us know on Slack or GitHub. #402

    • Harvest should have a Prometheus HTTP service discovery end-point to make it easier to add/remove pollers #575

    • Harvest should include a MetroCluster dashboard #539 Thanks @darthVikes for reporting

    • Harvest should collect Qtree and Quota metrics #522 Thanks @jmg011 for reporting and validating this works in your environment

    • SVM dashboard: Make NFS version a variable. SVM variable should allow selecting all SVMs for a cluster wide view #454

    • Harvest should monitor ONTAP chassis sensors #384 Thanks to @hashi825 for raising this issue and reviewing the pull request

    • Harvest cluster dashboard should include All option in dropdown for clusters #630 thanks @TopSpeed for raising this on Slack

    • Harvest should collect volume sis status #519 Thanks to @jmg011 for raising

    • Separate cDOT and 7-mode dashboards allowing each to change independently #489 #501 #547

    • Improve collector and object template merging and documentation #493 #555 Thanks @hashi825 for reviewing and suggesting improvements

    • Harvest should support label sets, allowing you to add additional key-value pairs to a poller's metrics#538

    • bin/grafana import should create a matching label and rewrite queries to use chained variable when using label sets #550

    • Harvest poller's should reuse their previous Prometheus port when restarted and be more deterministic about picking free ports #596 #595 Thanks to @cordelster for reporting

    • Improve instantiated systemd template by specifying user/group, requires, and moving Unix pollers to the end of the list. #643 Thanks to @mamoep for reporting and providing the changes! :sparkles:

    • Harvest's Docker container should use local conf directory instead of copying into image. Makes upgrade and changing template files easier. #511

    • Improve Disk dashboard by showing total number of disks by node and aggregate #583

    • Harvest 7-mode dashboards should be provisioned when using Docker Compose workflow #544

    • When upgrading, bin/harvest grafana import should add dashboards to a release-named folder so earlier dashboards are not overwritten #616

    • client_timeout should be overridable in object template files #563

    • Increase ZAPI client timeouts for volume and workloads objects #617

    • Doctor: Ensure that all pollers export to unique Prometheus ports #597

    • Improve execution performance of Harvest management commands :rocket: bin/harvest start|stop|restart #600

    • Include eight cDOT dashboards that use InfluxDB datasource #466. Harvest does not support InfluxDB dashboards for 7-mode. Thanks to @SamyukthaM for working on these

    • Docs: Describe how Harvest converts template labels into Prometheus labels #585

    • Docs: Improve Matrix documentation to better align with code #485

    • Docs: Improve ARCHITECTURE.md #603

    Fixes

    • Poller should report metadata when running on BusyBox #529 Thanks to @charz for reporting issue and providing details

    • Space used % calculation was incorrect for Cluster and Aggregate dashboards #624 Thanks to @faguayot and @jorbour for reporting.

    • When ONTAP indicates a counter is deprecated, but a replacement is not provided, continue using the deprecated counter #498

    • Harvest dashboard panels must specify a Prometheus datasource to correctly handles cases were a non-Prometheus default datasource is defined in Grafana. #639 Thanks for reporting @MrObvious

    • Prometheus datasource was missing on five dashboards (Network and Disk) #566 Thanks to @survive-wustl for reporting

    • Document permissions that Harvest requires to monitor ONTAP with a read-only user #559 Thanks to @survive-wustl for reporting and working with us to chase this down. :thumbsup:

    • Metadata dashboard should show correct status for running/stopped pollers #567 Thanks to @cordelster for reporting

    • Harvest should serve a human-friendly :corn: overview page of metric types when hitting the Prometheus end-point #613 Thanks @cordelster for reporting

    • SnapMirror plugin should include source_node #608

    • Disk dashboard should use better labels in table details #578

    • SVM dashboard should show correct units and remove duplicate graph #454

    • FCP plugin should work with 7-mode clusters #464

    • Node values are missing from some 7-mode perf counters #467

    • Nic state is missing from several network related dashboards 486

    • Reduce log noise when templates are not found since this is often expected #606

    • Use diagnosis-config-get-iter to collect node status from 7-mode systems #499

    • Node status is missing from 7-mode #527

    • Improve 7-mode templates. Remove cluster from 7-mode. yamllint all templates #531

    • When saving Grafana auth token, make sure bin/grafana writes valid Yaml #544

    • Improve Yaml parsing when different levels of indention are used in harvest.yml. You should see fewer invalid indentation messages. :clap: #626

    • Unix poller should ignore /proc files that aren't readable #249

    Source code(tar.gz)
    Source code(zip)
    harvest-21.11.0-1.amd64.deb(54.41 MB)
    harvest-21.11.0-1.x86_64.rpm(45.43 MB)
    harvest-21.11.0-1_linux_amd64.tar.gz(58.77 MB)
  • v21.08.0(Aug 31, 2021)

    Change Log

    Releases

    21.08.0 / 2021-08-31

    This major release introduces a Docker workflow that makes it a breeze to standup Grafana, Prometheus, and Harvest with auto-provisioned dashboards. There are seven new dashboards, example Prometheus alerts, and a bunch of fixes detailed below. We haven't forgotten about our 7-mode customers either and have a number of improvements in 7-mode dashboards with more to come.

    This release Harvest also sports the most external contributions to date. :metal: Thanks!

    With 284 commits since 21.05, there is a lot to summarize! Make sure you check out the full list of enhancements and improvements in the CHANGELOG.md since 21.05.

    IMPORTANT Harvest relies on the autosupport sidecar binary to periodically send usage and support telemetry data to NetApp by default. Usage of the harvest-autosupport binary falls under the NetApp EULA. Automatic sending of this data can be disabled with the autosupport_disabled option. See Autosupport for details.

    IMPORTANT RPM and Debian packages will be deprecated in the future, replaced with Docker and native binaries. See #330 for details and tell us what you think. Several of you have already weighed-in. Thanks! If you haven't, please do.

    IMPORTANT After upgrade don't forget to re-import all dashboards so you get new dashboard enhancements and fixes. You can re-import via bin/harvest/grafana cli or from the Grafana UI.

    Known Issues

    We've improved several of the 7-mode dashboards this release, but there are still a number of gaps with 7-mode dashboards when compared to c-mode. We will address these in a point release by splitting the c-mode and 7-mode dashboards. See #423 for details.

    On RHEL and Debian, the example Unix collector does not work at the moment due to the harvest user lacking permissions to read the /proc filesystem. See #249 for details.

    Enhancements

    • Make it easy to install Grafana, Prometheus, and Harvest with Docker Compose and auto-provisioned dashboards. #349

    • Lun, Volume Details, Node Details, Network Details, and SVM dashboards added to Harvest. Thanks to @jgasher for contributing five solid dashboards. :tada: #458 #482

    • Disk dashboard added to Harvest with disk type, status, uptime, and aggregate information. Thanks to @faguayot, @bengoldenberg, and @talshechanovitz for helping with this feature #348 #375 #367 #361

    • New SVM dashboard with NFS v3, v4, and v4.1 frontend drill-downs. Thanks to @burkl for contributing these. :tada: #344

    • Harvest templates should be extendible without modifying the originals. Thanks to @madhusudhanarya and @albinpopote for reporting. #394 #396 #391

    • Sort all variables in Harvest dashboards in ascending order. Thanks to @florianmulatz for raising #350

    • Harvest should include example Prometheus alerting rules #414

    • Improved documentation on how to send new ZAPIs and modify existing ZAPI templates. Thanks to @albinpopote for reporting. #397

    • Improve Harvest ZAPI template selection when monitoring a broader set of ONTAP clusters including 7-mode and 9.10.X #407

    • Collectors should log their full ZAPI request/response(s) when their poller includes a log section #382

    • Harvest should load config information from the HARVEST_CONF environment variable when set. Thanks to @ybizeul for reporting. #368

    • Document how to delete time series data from Prometheus #393

    • Harvest ZAPI tool supports printing results in XML and colors. This makes it easier to post-process responses in downstream pipelines #353

    • Harvest version should check for a new release and display it when available #323

    • Document how client authentication works and how to troubleshoot #325

    Fixes

    • ZAPI collector should recover after losing connection with ONTAP cluster for several hours. Thanks to @hashi825 for reporting this and helping us track it down #356

    • ZAPI templates with the same object name overwrite matrix data (impacted nfs and object_store_client_op templates). Thanks to @hashi825 for reporting this #462

    • Lots of fixes for 7-mode dashboards and data collection. Thanks to @madhusudhanarya and @ybizeul for reporting. There's still more work to do for 7-mode, but we understand some of our customers rely on Harvest to help them monitor these legacy systems. #383 #441 #423 #415 #376

    • Aggregate dashboard "space used column" should use current fill grade. Thanks to @florianmulatz for reporting. #351

    • When building RPMs don't compile Harvest Python test code. Thanks to @madhusudhanarya for reporting. #385

    • Harvest should collect include NVMe and fiber channel port counters. Thanks to @jgasher for submitting these. #363

    • Harvest should export NFS v4 metrics. It does for v3 and v4.1, but did not for v4 due to a typo in the v4 ZAPI template. Thanks to @jgasher for reporting. #481

    • Harvest panics when port_range is used in the Prometheus exporter and address is missing. Thanks to @ybizeul for reporting. #357

    • Network dashboard fiber channel ports (FCP) should report read and write throughput #445

    • Aggregate dashboard panel titles should match the information being displayed #133

    • Harvest should handle ZAPIs that include signed integers. Most ZAPIs use unsigned integers, but a few return signed ones. Thanks for reporting @hashi825 #384

    Source code(tar.gz)
    Source code(zip)
    harvest-21.08.0-6.amd64.deb(47.94 MB)
    harvest-21.08.0-6.x86_64.rpm(39.31 MB)
    harvest-21.08.0-6_linux_amd64.tar.gz(51.99 MB)
  • v21.05.4(Jul 22, 2021)

    Change Log

    Releases

    21.05.4 / 2021-07-22

    This release introduces Qtree protocol collection, improved Docker and client authentication documentation, publishing to Docker Hub, and a new plugin that helps build richer dashboards, as well as a couple of important fixes for collector panics.

    IMPORTANT RPM and Debian packages will be deprecated in the future, replaced with Docker and native binaries. See #330 for details and tell us what you think.

    Known Issues

    On RHEL and Debian, the example Unix collector does not work at the moment due to the harvest user lacking permissions to read the /proc filesystem. See #249 for details.

    Enhancements

    • Harvest collects Qtree protocol ops #298. Thanks to Martin Möbius for contributing

    • Harvest Grafana tool (optionally) adds a user-specified prefix to all Dashboard metrics during import. See harvest grafana --help #87

    • Harvest is taking its first steps to talk REST: query ONTAP, show Swagger API, model, and definitions #292

    • Tagged releases of Harvest are published to Docker Hub

    • Harvest honors Go's http(s) environment variable proxy information. See https://pkg.go.dev/net/http#ProxyFromEnvironment for details #252

    • New plugin value_to_num helps map labels to numeric values for Grafana dashboards. Current dashboards updated to use this plugin #319

    • harvest.yml supports YAML flow style. E.g. collectors: [Zapi] #260

    • New Simple collector that runs on Macos and Unix #270

    • Improve client certificate authentication documentation

    • Improve Docker deployment documentation 4019308

    Fixes

    • Harvest collector should not panic when resources are deleted from ONTAP #174 and #302. Thanks to @hashi825 and @mamoep for providing steps to reproduce

    • Shelf metrics should report on op-status for components. Thanks to @hashi825 for working with us on this fix and dashboard improvements #262

    • Harvest should not panic when InfluxDB is the only exporter #286

    • Volume dashboard space-used column should display with percentage filled. Thanks to @florianmulatz for reporting and suggesting a fix #303

    • Certificate authentication should honor path in harvest.yml #318

    • Harvest should not kill processes with poller in their arguments #328

    • Harvest ZAPI command line tool should limit perf-object-get-iter to subset of counters when using --counter #299


    Source code(tar.gz)
    Source code(zip)
    harvest-21.05.4-2.amd64.deb(37.18 MB)
    harvest-21.05.4-2.x86_64.rpm(30.91 MB)
    harvest-21.05.4-2_linux_amd64.tar.gz(40.87 MB)
  • v21.05.3(Jun 23, 2021)

    Change Log

    Releases

    21.05.3 / 2021-06-28

    This release introduces a significantly simplified way to connect Harvest and Prometheus, containerization enchantments, improved Harvest build times by 7x, reduced executable sizes by 3x, enabled cross compiling support, and several dashboard and other fixes.

    :tada: Thanks especially to @hashi825, @mamoep, @matejzero, and @florianmulatz for opening issues and pitching in to help fix them this release.

    Known Issues

    On RHEL and Debian, the example Unix collector does not work at the moment due to the harvest user lacking permissions to read the /proc filesystem. See #249 for details.

    Enhancements

    • Create Prometheus port range exporter that allows you to connect multiple pollers to Prometheus without needing to specify a port-per-poller. This makes it much easier to connect Prometheus and Harvest; especially helpful when you're monitoring many clusters #172

    • Improve Harvest build times by 7x and reduce executable sizes by 3x #100

    • Improve containerization with the addition of a poller-per-container Dockerfile. Create a new subcommand harvest generate docker which generates a docker-compose.yml file for all pollers defined in your config

    • Improve systemd integration by using instantiated units for each poller and a harvest target to tie them together. Create a new subcommand harvest generate systemd which generates a Harvest systemd target for all pollers defined in your config #systemd

    • Harvest doctor checks that all Prometheus exporters specify a unique port #118

    • Harvest doctor warns when an unknown exporter type is specified (likely a spelling error) #118

    • Add Harvest CUE validation and type-checking #208

    • bin/zapi uses the --config command line option to read the harvest config file. This brings this tool inline with other Harvest tools. This makes it easier to switch between multiple sets of harvest.yml files.

    • Harvest no longer writes pidfiles; simplifying management code and install #159

    Fixes

    • Ensure that the Prometheus exporter does not create duplicate labels #132

    • Ensure that the Prometheus exporter includes HELP and TYPE metatags when requested. Some tools require these #104

    • Disk status should return zero for a failed disk and one for a healthy disk. Thanks to @hashi825 for reporting and fixing #182

    • Lun info should be collected by Harvest. Thanks to @hashi825 for reporting and fixing #230

    • Grafana dashboard units, typo, and filtering fixes. Thanks to @mamoep, @matejzero, and @florianmulatz for reporting these :tada: #184 #186 #190 #192 #195 #202

    • Unix collector should not panic when harvest.yml is changed #160

    • Reduce log noise about poller lagging behind by few milliseconds. Thanks @hashi825 #214

    • Don't assume debug when foregrounding the poller process. Thanks to @florianmulatz for reporting. #246

    • Improve Docker all-in-one-container argument handling and simplify building in air gapped environments. Thanks to @optiz0r for reporting these issues and creating fixes. #166 #167 #168

    Source code(tar.gz)
    Source code(zip)
    harvest-21.05.3-2.amd64.deb(30.40 MB)
    harvest-21.05.3-2.x86_64.rpm(25.09 MB)
    harvest-21.05.3-2_linux_amd64.tar.gz(33.47 MB)
  • v21.05.2(Jun 14, 2021)

    Change Log

    Releases

    21.05.2 / 2021-06-14

    This release adds support for user-defined URLs for InfluxDB exporter, a new command to validate your harvest.yml file, improved logging, panic handling, and collector documentation. We also enabled GitHub security code scanning for the Harvest repo to catch issues sooner. These scans happen on every push.

    There are also several quality-of-life bug fixes listed below.

    Fixes

    • Handle special characters in cluster credentials #79
    • TLS server verification works with basic auth #51
    • Collect metrics from all disk shelves instead of one #75
    • Disk serial number and is-failed are missing from cdot query #60
    • Ensure collectors and pollers recover from panics #105
    • Cluster status is initially reported, but then stops being reported #66
    • Performance metrics don't display volume names #40
    • Allow insecure Grafana TLS connections --insecure and honor requested transport. See harvest grafana --help for details #111
    • Prometheus dashboards don't load when exemplar is true. Thanks to @sevenval-admins, @florianmulatz, and @unbreakabl3 for their help tracking this down and suggesting a fix. #96
    • harvest stop does not stop pollers that have been renamed #20
    • Harvest stops working after reboot on rpm/deb #50
    • harvest start shall start as harvest user in rpm/deb #129
    • harvest start detects stale pidfiles and makes start idempotent #123
    • Don't include unknown metrics when talking with older versions of ONTAP #116

    Enhancements

    • InfluxDB exporter supports user-defined URLs
    • Add workload counters to ZapiPerf #9
    • Add new command to validate harvest.yml file and optionally redact sensitive information #16 e.g. harvest doctor --config ./harvest.yml
    • Improve documentation for Unix, Zapi, and ZapiPerf collectors
    • Add Zerolog framework for structured logging #61
    • Vendor 3rd party code to increase reliability and make it easier to build in air-gapped environments #26
    • Make contributing easier with a digital CCLA instead of 1970's era PDF :)
    • Enable GitHub security code scanning
    • InfluxDB exporter provides the option to pass the URL end-point unchanged. Thanks to @steverweber for their suggestion and validation. #63
    Source code(tar.gz)
    Source code(zip)
    harvest-21.05.2-1.amd64.deb(68.21 MB)
    harvest-21.05.2-1.x86_64.rpm(74.93 MB)
    harvest-21.05.2-1_linux_amd64.tar.gz(75.28 MB)
  • v21.05.1(May 20, 2021)

    Change Log

    Releases

    21.05.1 / 2021-05-20

    Announcing the release of Harvest2. With this release the core of Harvest has been completely rewritten in Go. Harvest2 is a replacement for the older versions of Harvest 1.6 and below.

    If you're using one of the Harvest 2.x release candidates, you can do a direct upgrade.

    Going forward Harvest2 will follow a year.month.fix release naming convention with the first release being 21.05.0. See SUPPORT.md for details.

    IMPORTANT v21.05 increased Harvest's out-of-the-box security posture - self-signed certificates are rejected by default. You have two options:

    1. Setup client certificates for each cluster
    2. Disable the TLS check in Harvest. To disable, you need to edit harvest.yml and add use_insecure_tls=true to each poller or add it to the Defaults section. Doing so tells Harvest to ignore invalid TLS certificates.

    IMPORTANT RPM and Debian packages will be deprecated in the future, replaced with Docker and native packages.

    IMPORTANT Harvest 1.6 is end of support. We recommend you upgrade to Harvest 21.05 to take advantage of the improvements.

    Changes since rc2

    Fixes

    • Log mistyped exporter names and continue, instead of stopping
    • harvest grafana should work with custom harvest.yml files passed via --config
    • Harvest will try harder to stop pollers when they're stuck
    • Add Grafana version check to ensure Harvest can talk to a supported version of Grafana
    • Normalize rate counter calculations - improves latency values
    • Workload latency calculations improved by using related objects operations
    • Make cli flags consistent across programs and subcommands
    • Reduce aggressive logging; if first object has fatal errors, abort to avoid repetitive errors
    • Throw error when use_insecure_tls is false and there are no certificates setup for the cluster
    • Harvest status fails to print port number after restart
    • RPM install should create required directories
    • Collector now warns if it falls behind schedule
    • package.sh fails without internet connection
    • Version flag is missing new line on some shells #4
    • Poller should not ignore --config #28

    Enhancements

    • Add new exporter for InfluxDB
    • Add native install package
    • Add ARCHITECTURE.md and improve overall documentation
    • Use systemd harvest.service on RPM and Debian installs to manage Harvest
    • Add runtime profiling support - off by default, enabled with --profiling flag. See harvest start --help for details
    • Document how to use ONTAP client certificates for password-less polling
    • Add per-poller Prometheus end-point support with promPort
    • The release, commit and build date information are baked into the release executables
    • You can pick a subset of pollers to manage by passing the name of the poller to harvest. e.g. harvest start|stop|restart POLLERS
    Source code(tar.gz)
    Source code(zip)
    harvest-21.05.1-1.amd64.deb(58.03 MB)
    harvest-21.05.1-1.tar.gz(63.06 MB)
    harvest-21.05.1-1.x86_64.rpm(62.66 MB)
  • v21.05.0(May 20, 2021)

    Change Log

    Releases

    21.05.0 / 2021-05-20

    Announcing the release of Harvest2. With this release the core of Harvest has been completely rewritten in Go. Harvest2 is a replacement for the older versions of Harvest 1.6 and below.

    If you're using one of the Harvest 2.x release candidates, you can do a direct upgrade.

    Going forward Harvest2 will follow a year.month.fix release naming convention with the first release being 21.05.0. See SUPPORT.md for details.

    IMPORTANT v21.05 increased Harvest's out-of-the-box security posture - self-signed certificates are rejected by default. You have two options:

    1. Setup client certificates for each cluster
    2. Disable the TLS check in Harvest. To disable, you need to edit harvest.yml and add use_insecure_tls=true to each poller or add it to the Defaults section. Doing so tells Harvest to ignore invalid TLS certificates.

    IMPORTANT RPM and Debian packages will be deprecated in the future, replaced with Docker and native packages.

    IMPORTANT Harvest 1.6 is end of support. We recommend you upgrade to Harvest 21.05 to take advantage of the improvements.

    Changes since rc2

    Fixes

    • Log mistyped exporter names and continue, instead of stopping
    • harvest grafana should work with custom harvest.yml files passed via --config
    • Harvest will try harder to stop pollers when they're stuck
    • Add Grafana version check to ensure Harvest can talk to a supported version of Grafana
    • Normalize rate counter calculations - improves latency values
    • Workload latency calculations improved by using related objects operations
    • Make cli flags consistent across programs and subcommands
    • Reduce aggressive logging; if first object has fatal errors, abort to avoid repetitive errors
    • Throw error when use_insecure_tls is false and there are no certificates setup for the cluster
    • Harvest status fails to print port number after restart
    • RPM install should create required directories
    • Collector now warns if it falls behind schedule
    • package.sh fails without internet connection
    • Version flag is missing new line on some shells #4

    Enhancements

    • Add new exporter for InfluxDB
    • Add native install package
    • Add ARCHITECTURE.md and improve overall documentation
    • Use systemd harvest.service on RPM and Debian installs to manage Harvest
    • Add runtime profiling support - off by default, enabled with --profiling flag. See harvest start --help for details
    • Document how to use ONTAP client certificates for password-less polling
    • Add per-poller Prometheus end-point support with promPort
    • The release, commit and build date information are baked into the release executables
    • You can pick a subset of pollers to manage by passing the name of the poller to harvest. e.g. harvest start|stop|restart POLLERS
    Source code(tar.gz)
    Source code(zip)
    harvest-21.05.0-1.amd64.deb(58.00 MB)
    harvest-21.05.0-1.tar.gz(63.06 MB)
    harvest-21.05.0-1.x86_64.rpm(62.65 MB)
Owner
NetApp
We unlock the best of cloud.
NetApp
Sensu-go-postgres-metrics - The sensu-go-postgres-metrics is a sensu check that collects PostgreSQL metrics

sensu-go-postgres-metrics Table of Contents Overview Known issues Usage examples

Scott Cupit 0 Jan 12, 2022
The metrics-agent collects allocation metrics from a Kubernetes cluster system and sends the metrics to cloudability

metrics-agent The metrics-agent collects allocation metrics from a Kubernetes cluster system and sends the metrics to cloudability to help you gain vi

null 0 Jan 14, 2022
Observe host metrics collector Written in Go

Observe Host Metrics Collector Experimental collector and forwarder of host metr

Max Skybin 0 Jan 4, 2022
Metrics collector and ebpf-based profiler for C, C++, Golang, and Rust

Apache SkyWalking Rover SkyWalking Rover: Metrics collector and ebpf-based profiler for C, C++, Golang, and Rust. Documentation Official documentation

The Apache Software Foundation 53 Aug 1, 2022
Go http.RoundTripper that emits open telemetry metrics. This helps you easily get metrics for all external APIs you interact with.

go-otelroundtripper This package provides an easy way to collect http related metrics (e.g Response times, Status Codes, number of in flight requests

Ndole Studio 59 Jul 29, 2022
AppsFlyer 487 Jul 16, 2022
Shopping List webapp with Prometheus metrics endpoint

shopping-list Shopping List webapp with Prometheus metrics endpoint This is an example app with prometheus metrics integrated. This app runs on port 5

Lev Shvarts 0 Nov 4, 2021
Prometheus metrics for Go database/sql via VictoriaMetrics/metrics

sqlmetrics Prometheus metrics for Go database/sql via VictoriaMetrics/metrics Features Simple API. Easy to integrate. Install Go version 1.16+ go get

cristaltech 22 Jul 14, 2022
cluster-api-state-metrics (CASM) is a service that listens to the Kubernetes API server and generates metrics about the state of custom resource objects related of Kubernetes Cluster API.

Overview cluster-api-state-metrics (CASM) is a service that listens to the Kubernetes API server and generates metrics about the state of custom resou

Daimler Group 58 Jul 23, 2022
AutoK3s GEO collects metrics about locates remote IP-address and exposes metrics to InfluxDB.

AutoK3s GEO AutoK3s GEO collects metrics about locates remote IP-address and exposes metrics to InfluxDB. Thanks to https://freegeoip.live/ which prov

Jason 0 Jun 16, 2022
Flash-metrics - Flash Metrics Storage With Golang

Flash Metrics Storage bootstrap: $ echo -e "max-index-length = 12288" > tidb.con

null 3 Jan 8, 2022
This library provides a metrics package which can be used to instrument code, expose application metrics, and profile runtime performance in a flexible manner.

This library provides a metrics package which can be used to instrument code, expose application metrics, and profile runtime performance in a flexible manner.

null 0 Jan 18, 2022
A CLI tool that generates OpenTelemetry Collector binaries based on a manifest.

OpenTelemetry Collector builder This program generates a custom OpenTelemetry Collector binary based on a given configuration. TL;DR $ go get github.c

OpenTelemetry - CNCF 50 Jul 4, 2022
File Collector is an application that uses HTTP protocol to collect homework and files.

File Collector File Collector is an application that uses HTTP protocol to collect homework and files. Usage When filecollector is run, it will automa

毛亚琛 1 Jun 16, 2022
This application is a NetFlow/IPFIX/sFlow collector in Go.

GoFlow This application is a NetFlow/IPFIX/sFlow collector in Go. It gathers network information (IP, interfaces, routers) from different flow protoco

Alireza Rostami 3 Mar 7, 2022
Go-based search engine URL collector , support Google, Bing, can be based on Google syntax batch collection URL

Go-based search engine URL collector , support Google, Bing, can be based on Google syntax batch collection URL

Re 52 Jul 20, 2022
This application is a NetFlow/IPFIX/sFlow collector in Go.

GoFlow This application is a NetFlow/IPFIX/sFlow collector in Go. It gathers network information (IP, interfaces, routers) from different flow protoco

null 2 Mar 11, 2022
This POC is built with the goal to collect events/logs from the host systems such as Kubernetes, Docker, VMs, etc. A buffering layer is added to buffer events from the collector

What is does This POC is build with the goal to collect events/logs from the host systems such as Kubernetes, docker, VMs etc. A buffering layer is ad

Gufran  Mirza 3 Dec 16, 2021
Alibaba iLogtail : The Lightweight Collector of SLS in Alibaba Cloud

Alibaba iLogtail - The Lightweight Collector of SLS in Alibaba Cloud | 中文版本 iLogtail was born for observable scenarios and has many production-level f

Alibaba 783 Aug 3, 2022
A collector api for golang

Install go 1.17 curl https://go.dev/dl/go1.17.3.linux-amd64.tar.gz tar -C /usr/local -xzf go1.17.3.linux-amd64.tar.gz Build for alpine container CGO_

Christophe Varoqui 0 Dec 28, 2021
Enterprise Network Flow Collector (IPFIX, sFlow, Netflow)

High-performance, scalable and reliable IPFIX, sFlow and Netflow collector (written in pure Golang). Features IPFIX RFC7011 collector sFLow v5 raw hea

Edgecast 911 Jul 21, 2022
System information collector

Gohai Gohai is a tool which collects an inventory of system information. It aims to implement some parts of features from facter and ohai. It's forked

null 0 Mar 19, 2020
The stats collector for CSC demo files

demoScrape2 The stats collector for CSC demo files. Join the CSC discord

null 1 Jan 8, 2022
Coletor-mpac - Collector of the Public Ministry of Acre (MPAC) using CDP technology

Ministério Público do Acre(MPAC) Esse coletor é baseado na tecnologia Chrome Dev

DadosJusBR 1 Jan 28, 2022
The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.

The open-source platform for monitoring and observability. Grafana allows you to query, visualize, alert on and understand your metrics no matter wher

Grafana Labs 50.2k Jul 29, 2022
Vilicus is an open source tool that orchestrates security scans of container images(docker/oci) and centralizes all results into a database for further analysis and metrics.

Vilicus Table of Contents Overview How does it work? Architecture Development Run deployment manually Usage Example of analysis Overview Vilicus is an

Ederson Brilhante 77 Jun 21, 2022
Pokes users on Slack about outstanding risks found by Crowdstrike Spotlight or vmware Workspace ONE so they can secure their own endpoint.

?? security-slacker Pokes users on Slack about outstanding risks found by Crowdstrike Spotlight or vmware Workspace ONE so they can secure their own e

Niels Hofmans 20 Jun 13, 2022
kubetnl tunnels TCP connections from within a Kubernetes cluster to a cluster-external endpoint, e.g. to your local machine. (the perfect complement to kubectl port-forward)

kubetnl kubetnl (kube tunnel) is a command line utility to tunnel TCP connections from within a Kubernetes to a cluster-external endpoint, e.g. to you

null 4 Nov 16, 2021
rtsp to webrtc proxy with websocket signaling, currently limited to single h264 stream per endpoint

rtp-to-webrtc rtp-to-webrtc demonstrates how to consume a RTP stream video UDP, and then send to a WebRTC client. With this example we have pre-made G

Game On 4 May 12, 2022