k6 v0.42.0 is here! :tada:

This release includes:

• A tiny breaking change to improve WebSockets response handling.
• A new experimental output.
• More features in our experimental WebSocket module.
• Wildcard support for hosts.
• Some bug fixes, UX improvements, and maintenance.

Breaking changes

• #2712 k6/ws returns an HTTP response for failed connections instead of an undefined behavior. Thanks, @brietaylor.

New Features

Experimental Prometheus Remote Write Output #2784

This release brings a new builtin Output to any Prometheus Remote Write implementations (e.g. Prometheus, Mimir). This is an experimental feature, and future releases could introduce breaking changes.

The following example uses k6 run with the new output. It uses the defaults options, such as the Remote Write server URL (http://localhost:9090/api/v1/write):

k6 run -o experimental-prometheus-rw script.js

It supports the new and convenient experimental Native Histogram feature, added in Prometheus v2.40.0. It's not enabled by default, but we expect to make it the default way to map k6 Trend metrics once the Prometheus project signals that its mature enough and when more Remote Write implementations support it. For now, if you want to use it, you need to set the environment variable K6_PROMETHEUS_RW_TREND_AS_NATIVE_HISTOGRAM to true.

You can find complete documentation with more examples, use cases, and available configurations.

More features for the experimental websockets module #2786

The k6/experimental/websockets module that we announced in the v0.40.0 release got an update that extends its functionality.

It brings some useful features that the k6/ws module already has, like cookies, custom headers, compression and tags customization support, the syntax to define event handlers (onopen, onmessage, etc.) and ping/pong functionality.

This is still an experimental module, but with the recent changes we think it's usable for most users. So whether you're writing a new WebSocket test, or currently using the k6/ws module, we invite you to give it a try, and report any issues in the project's issue tracker.

import { WebSocket } from "k6/experimental/websockets";
import {
  setTimeout,
  clearTimeout,
  setInterval,
  clearInterval
} from "k6/experimental/timers";

const CLOSED_STATE = 3

export default function () {
  const params = {
    "tags": {
      "my_tag": "hello"
    }
  };

  const ws = new WebSocket('ws://localhost:10000', null, params);

  ws.onopen = () => {
    console.log('connected');
    ws.send(Date.now().toString());
  };

  let intervalId = setInterval(() => {
    ws.ping();
    console.log("Pinging every 1 sec (setInterval test)");
  }, 1000);

  let timeout1id = setTimeout(function () {
    console.log('2 seconds passed, closing the socket');
    clearInterval(intervalId);
    ws.close();
  }, 2000);

  ws.onclose = () => {
    clearTimeout(timeout1id);
    console.log('disconnected');
  };

  ws.onping = () => {
    console.log("PING!");
  };

  ws.onpong = () => {
    console.log("PONG!");
  };

  // Multiple event handlers on the same event
  ws.addEventListener("pong", () => {
    console.log("OTHER PONG!");
  });

  ws.onmessage = (m) => {
    let parsed = parseInt(m.data, 10)
    if (Number.isNaN(parsed)) {
      console.log('Not a number received: ', m.data);
      return
    }

    console.log(`Roundtrip time: ${Date.now() - parsed} ms`);

    let timeoutId = setTimeout(function () {
      if (ws.readyState == CLOSED_STATE) {
        console.log("Socket closed, not sending anything");

        clearTimeout(timeoutId);
        return;
      }

      ws.send(Date.now().toString());
    }, 500);
  };

  ws.onerror = (e) => {
    if (e.error != "websocket: close sent") {
      console.log('An unexpected error occurred: ', e.error);
    }
  };
};

The module docs has a complete reference, and some examples.

Wildcard support for hosts option #2747

Thanks to the great effort from @eugercek, the hosts option now accepts domains that contain a wildcard at the beginning. It can be helpful for setting multiple subdomains of the same domain, so instead of setting subdomain1.k6.io': '1.2.3.4', 'subdomain2.k6.io': '1.2.3.4' it is possible to use the wildcard for setting directly *.k6.io: '1.2.3.4'.

export const options = {
  hosts: {
    '*.k6.io': '1.2.3.4',
  },
}

Enhancements and UX improvements

• #2660 Pre-loads the operating system TLS certificates. Thanks, @tbourrely.
• #2791 Initializes VUs for setup() and teardown() only if they are defined in the script.

Bug fixes

• #2759 Ensures the evaluation of thresholds over trend metrics' median.
• #2759 Fixes a few potential Output data races for interrupted test runs.
• #2767 Fixes the emission of ws_session_duration when setup throws an error.
• #2773 Ensures that JavaScript runtime makes only one copy of the exports for each module including built-in ones.

Maintenance and internal improvements

We had a few minor changes in this release:

• #2757 goja runtime has been updated.
• #2768 WS.Connect() has been refactored.
• #2770 Refactored parts of the js module.
• #2782 Covered more relative path test cases for require and open.
• #2789, #2792, #2795, #2796 Improved stability of the integration tests.
• #2791 Optimized the performance of the internal Trie implementation. Thanks, @eugercek.

Full Changelog: https://github.com/grafana/k6/compare/v0.41.0...v0.42.0

k6 v0.41.0 is here! :tada: It has relatively few user-facing changes, but includes massive internal improvements that pave the way for some awesome features for the near future. Unfortunately, this work also required us to make a few minor breaking changes.

Breaking changes

Changes in the url, iter and vu system metric tags

As we warned in the release notes for k6 v0.39.0 and v0.40.0, we've been fundamentally refactoring the metrics sub-systems of k6. We now have efficient support for time series, which required a few minor user-facing breaking changes:

• If URL grouping is used for HTTP requests (that is, if the http.url helper is used or the name metric tag is specified), then the url tag in the resulting http_req_* metric samples will also have the same value as the name tag. Previously, k6 did this only for the cloud output, but now it does this universally (#2703).
• The vu and iter system tags, which are disabled by default, have been transformed into high-cardinality metrics metadata instead. It means that they will no longer be usable in thresholds, and various outputs may emit them differently or ignore them completely (#2726).

Changes in the Go metrics APIs

While the user-facing changes from our metrics refactoring are few and relatively minor, and there are no changes to JavaScript APIs yet, we have extensively refactored our internal Go APIs (#2594, #2726, #2727). The metrics.Sample and metrics.TagSet types are now entirely different. We also have high-cardinality metadata attributes in each Sample and at the VU level (see the combined TagsAndMeta code and how it is used in the per-VU State object).

k6 convert is officially deprecated (#2714)

k6 convert has been a sub-command to convert a HAR file recording of HTTP traffic into a preliminary k6 script that makes roughly the same requests. It has been long neglected and softly deprecated in favor of the newer and more feature-rich har-to-k6 standalone converter.

We have now officially deprecated k6 convert. The command still works and will continue to do so for a few more k6 versions. However, it's not visible from k6 --help and will emit a warning when used. Please see the documentation for the standalone har-to-k6 converter and open an issue (or comment on an existing one) if you have any problems with it.

New Features, enhancements, and UX improvements

• #2679 added support for maxReceiveSize and maxSendSize parameters in the gRPC's Client.connect() method. Thanks, @ariasmn!
• #2605 introduced a new --exclude-env-vars CLI flag to k6 archive that causes it to not include the provided environment variables in the resulting archive bundle's metadata.json file.
• #2700 added support for loading gRPC protoset files. Thanks, @jklipp!

Bug fixes

• #2678 fixed the Docker image labels. Thanks, @knittl, for reporting the problem (#2677)!
• #2689 fixed the REST API's Content-Type response header. Thanks, @wingyplus!
• #2691 fixed the detailed k6 version information embedded in the k6 releases.
• #2693 fixed a bug that made the k6 event loop unusable when a Promise rejection was left unhandled.
• #2696 fixed a problem with HTTP redirects with empty Location headers (#2474) by updating the Go version we use to compile k6 to 1.19.x. Thanks, @agilob!
• #2705 fixed a panic in the k6/net/grpc module (#2661). Thanks, @c47gao and @robpickerill!
• #2738 fixed a panic when a Promise was rejected with an undefined reason.
• #2739 fixed hidden stack traces in certain types of errors originating from k6's Go code.

Maintenance and internal improvements

We had a few minor changes in this release:

• #2687 improved our logging tests. Thanks, @nsmith5!
• #2696 updated the used Go version to 1.19.x and the Alpine version in our Docker image to 3.16. Thanks, @agilob!
• #2707, #2708, #2709, #2710 updated most of the Go dependencies k6 has.
• #2716 refactored how custom JS tags are applied to metrics and cleaned up validation for invalid tag values.

We also have a couple of significant improvements that will help us develop exciting new features soon:

Metric time series (#2594)

Previous to #2594, k6 didn't have an efficient way to group metric samples with the same tags. It meant that a whole class of applications for processing and aggregating metrics were nearly impossible to do or, at best, very inefficient.

At the cost of some minor breaking changes, we now have a performant internal representation to group metric samples with the same tags at the time of the action that generated them, i.e. the time of metric measurement. With this, k6 can efficiently group samples for the same action (e.g. an HTTP request to a specific URL) over time and construct time series with them.

Internal support for high-cardinality metrics metadata (#2726, #2727)

As described in the previous section, the efficient grouping of metric samples into time series works well for relatively low-cardinality data. However, k6 needed some way to attach high-cardinality metadata as well. This is necessary for data that's unique or random, such as Trace and Span IDs in distributed tracing or user IDs in tests with huge data sets.

k6 v0.41.0 has added support for attaching high-cardinality metadata to metric samples, and the vu and iter system tags have been transformed into such metadata (see the breaking changes section above), but it is not yet accessible from user scripts. There is no JavaScript API to modify this metadata, only built-in k6 Go modules and xk6 Go extensions can make use of it, for now.

k6 v0.40.0 is here! This release includes:

• Breaking changes to some undocumented and unintentional edge behaviors.
• New experimental modules and first-class support for JavaScript classes.
• Bugs and refactorings to pave the way for future features.

Finally, the Roadmap goes over the plans for the next cycles.

Breaking changes

• #2632 During the refactoring to set tags to metric.add in the order they are provided, we discovered that you could provide tags as a key-value pair map multiple times in the same call. This was never the intended use and was never documented. As it was undocumented, and as such functionality makes no sense alongside every other API k6 has, we decided to remove this ability.
• #2582 [For extensions using the event loop] Previously, if RegisterCallback result was called twice, the second call would silently break the event loop. This has never been expected behavior, and calling it twice is always a bug in the code using it. Now, calling the RegisterCallback result twice leads to a panic.
• #2596 The tainted property of the Metric type is no longer outputted by the JSON output. That property was likely always going to have a false value as it was outputted at the beginning of the test.

Main module/script no longer pollutes the global scope #2571

During the ESM changes, we found that anything defined in the main module scope was also accessible globally. This was because it was directly evaluated in the global scope. This has now been remedied and is no longer the case. This is a breaking change, but given that the whole point of modules (CommonJS or ESM) is to separate them, this is obviously rather a bug than a feature.

On that note, we've seen reports by people who have this global accessibility of the main module (intentionally or not). Still, it seems relatively rare, with only a few usages in a script. So if you need to access something globally, our suggested workaround is to set it explicitly on the global object globalThis.

k6/ws now respects the throw option #2247

k6/http has used the throw option to figure out whether it should throw an exception on errors or return a response object with an error set on it (and log it).

This functionality is finally also available for k6/ws, which previously would've always thrown an exception and thus involved more scripting in handling it (#2616).

This is a minor breaking change. By default, throw is false, so it now no longer throws an exception but instead returns a Response with error property.

Thank you, @fatelei, for making this change!

New Features

Experimental modules #2630 and #2656

As mentioned in the v0.39.0 release notes, we're happy to announce that this release brings experimental modules. The main idea behind this initiative is to get community feedback earlier, which will help us improve them faster. We encourage you to try experimental modules out and provide feedback through the community forums or GitHub issues.

This release contains three experimental modules:

• k6/experimental/redis - support for interaction with Redis
• k6/experimental/websockets - a new Websockets API that copies the "web" WebSocket API
• k6/experimental/timers - adding setTimeout/clearTimeout and setInterval/clearInterval implementations.

Important to highlight that the k6 team does not guarantee backward compatibility for these modules and may change or remove them altogether. Also, their import paths, starting with k6/experimental, will break when the modules stop being experimental. Of course, we are going to try to limit those breaking changes to a minimum and, when possible, do them in a backward compatible way for at least a version.

import { check } from "k6";
import http from "k6/http";
import redis from "k6/experimental/redis"; // this will be `k6/x/redis` if you are using it as extension
import { textSummary } from "https://jslib.k6.io/k6-summary/0.0.1/index.js";

export const options = {
  scenarios: {
    usingRedisData: {
      executor: "shared-iterations",
      vus: 10,
      iterations: 200,
      exec: "measureUsingRedisData",
    },
  },
};

// Instantiate a new redis client
const redisClient = new redis.Client({
  addrs: __ENV.REDIS_ADDRS.split(",") || new Array("localhost:6379"), // in the form of "host:port", separated by commas
  password: __ENV.REDIS_PASSWORD || "",
});

// Prepare an array of crocodile ids for later use
// in the context of the measureUsingRedisData function.
const crocodileIDs = new Array(0, 1, 2, 3, 4, 5, 6, 7, 8, 9);

export function setup() {
  redisClient.sadd("crocodile_ids", ...crocodileIDs);
}

export function measureUsingRedisData() {
  // Pick a random crocodile id from the dedicated redis set,
  // we have filled in setup().
  redisClient
    .srandmember("crocodile_ids")
    .then((randomID) => {
      const url = `https://test-api.k6.io/public/crocodiles/${randomID}`;
      const res = http.get(url);

      check(res, {
        "status is 200": (r) => r.status === 200,
        "content-type is application/json": (r) =>
          r.headers["Content-Type"] === "application/json",
      });

      return url;
    })
    .then((url) => redisClient.hincrby("k6_crocodile_fetched", url, 1));
}

export function teardown() {
  redisClient.del("crocodile_ids");
}

export function handleSummary(data) {
  redisClient
    .hgetall("k6_crocodile_fetched")
    .then((fetched) => Object.assign(data, { k6_crocodile_fetched: fetched }))
    .then((data) =>
      redisClient.set(`k6_report_${Date.now()}`, JSON.stringify(data))
    )
    .then(() => redisClient.del("k6_crocodile_fetched"));

  return {
    stdout: textSummary(data, { indent: "  ", enableColors: true }),
  };
}

The extension does not run a Redis server. You need to separately handle running, scaling, and connecting infrastructure to Redis.

The xk6-redis repository has more examples, and the module is documented in the official k6 documentation.

import { randomString, randomIntBetween } from "https://jslib.k6.io/k6-utils/1.1.0/index.js";
import { WebSocket } from "k6/experimental/websockets"
import { setTimeout, clearTimeout, setInterval, clearInterval } from "k6/experimental/timers"

let chatRoomName = 'publicRoom'; // choose your chat room name
let sessionDuration = randomIntBetween(5000, 60000); // user session between 5s and 1m


export default function() {
  for (let i = 0; i < 4; i++) {
    startWSWorker(i)
  }
}

function startWSWorker(id) {
  let url = `wss://test-api.k6.io/ws/crocochat/${chatRoomName}/`;
  let ws = new WebSocket(url);
  ws.addEventListener("open", () => {
    ws.send(JSON.stringify({ 'event': 'SET_NAME', 'new_name': `Croc ${__VU}:${id}` }));

    ws.addEventListener("message", (e) => {
      let msg = JSON.parse(e.data);
      if (msg.event === 'CHAT_MSG') {
        console.log(`VU ${__VU}:${id} received: ${msg.user} says: ${msg.message}`)
      }
      else if (msg.event === 'ERROR') {
        console.error(`VU ${__VU}:${id} received:: ${msg.message}`)
      }
      else {
        console.log(`VU ${__VU}:${id} received unhandled message: ${msg.message}`)
      }
    })


    let intervalId = setInterval(() => {
      ws.send(JSON.stringify({ 'event': 'SAY', 'message': `I'm saying ${randomString(5)}` }));
    }, randomIntBetween(2000, 8000)); // say something every 2-8seconds


    let timeout1id = setTimeout(function() {
      clearInterval(intervalId)
      console.log(`VU ${__VU}:${id}: ${sessionDuration}ms passed, leaving the chat`);
      ws.send(JSON.stringify({ 'event': 'LEAVE' }));
    }, sessionDuration);

    let timeout2id = setTimeout(function() {
      console.log(`Closing the socket forcefully 3s after graceful LEAVE`);
      ws.close();
    }, sessionDuration + 3000);

    ws.addEventListener("close", () => {
      clearTimeout(timeout1id);
      clearTimeout(timeout2id);
      console.log(`VU ${__VU}:${id}: disconnected`);
    })
  });
}

Note that no k6 iterations finish if any WebSocket is still open or if a timeout or an interval is not cleared or triggered. This means that your script must take care of clearing all intervals and closing the WebSocket at some point. However, k6 still kills the whole process if it takes too long to stop after the maximum test duration is reached.

Current issues and future improvements for the WebSockets API can be found in its issue tracker. Currently, documentation is available through MDN, though some features are not yet supported:

• no Blob binary type - ArrayBuffer is the default
• no onMessage and co. - only addEventListener

First-class support for JavaScript Classes

As part of updating goja, k6 got native support for classes. Again, that's native, as in not by transpiling by the internal Babel.

Because this actually implements classes as described in the latest ECMAScript specification, this also means we get a ton of additional class features that were never previously supported (for example, private fields). Additionally, at least one bug #1763 was fixed as a result of this, but probably many more as well.

Due to this fairly significant change, some code could behave differently. Please report any issues, though consider that it's possible that the new behavior is just the correct one.

Other updates from goja are:

• optimizations around using strings and some access patterns
• support for \u{01234} Unicode point syntax in regexp
• Fixed a case where interrupting the VM did not work, especially around try/catch usage (#2600). This was particularly problematic for k6, as it could lead to k6 hanging.

Many thanks to @dop251 for continuing to improve goja!

New Test runtime for module extension developers #2598

While we develop extensions internally, we often need to repeatedly create the same structure. With the addition of the event loops, it is now required to set it up as well. Even k6 team members get parts of this wrong every once in a while, so we added a small type to be used by (extension) module developers to write tests easier (#2602).

This API will likely change and evolve as we add more functionality or as we change the k6 internal API.

Bug fixes

• #2585 http.batch() now displays an error if it is not given exactly 1 argument(#1289). Thanks, @vanshaj!
• #2596 Fixes a potential data race in the JSON output. Includes a breaking change where tainted property is no longer outputted. That property was (likely) always going to have the value false as it was outputted at the beginning of the test.
• #2604 Fixes SSL keylogger not working with absolute paths.
• #2637 Fixes setting the options rps to 0 or below leading to exceptions. Now setting it to 0 or below disables the limit. Thanks, @tbourrely. #2613
• #2278 Reading options.tags directly was not possible. This was fixed by accident by #2631. k6/execution is still the recommended way to access the final options of the test.

Maintenance and internal improvements

• #2590 Updates direct dependencies without any interesting changes apart goja.
• #2591 Changes to the CI process to always build rpm/deb and windows packages and use nfpm to do it.
• #2593 Internal cleanup after finally removing common.Bind.
• #2597 Fixes go benchmarks we have broken over time.
• #2599 Reformats //nolint comments as part of getting ready for go 1.19.
• A bunch of fixes for tests #2589, #2620, #2625, #2643, #2647, #2648,
• #2607 Fixes the build badge in the README. Thanks @AetherUnbound!
• #2614 Fixes advice for RPM install on Amazon Linux.
• #2615 Improves documentation of the RegisterCallback, following feedback on how hard it was to understand.
• #2627 Create distinct test state objects for the pre-init and run phases.
• #2635 Drop License header in each file.
• #2636 Add SECURITY.md with instructions how to report security issues responsibly.
• #2641 Fix spelling of lose. Thanks @spazm!
• Update to golangci-lint v1.47.2 and enable a bunch more linters. #2609, #2611. Also, drop obsolete configurations #2619.

Roadmap and future plans

This section discusses our plans for future versions. Notice that two big ticket items are here again―ESM modules and metric refactoring. They remain on the roadmap mostly for the sheer size of the work required on both, but also for some unforeseen needed changes, which actually should make them better in the long run. It also so happens that it is vacation season so the k6 team rightfully is taking some personal time away.

Native support for ECMAScript modules

Native ESM support is coming. A PR to k6 is under development, and there's a branch that will become a PR to goja to add the support there. The k6 team is hopeful that this will land in the next version, v0.41.0!

It turned out that there were a lot more things to be added as functionality - dynamic import, and the tc39 group released the latest ECMAScript specification adding support of top-level await in modules. While neither k6 nor goja has support for the async/await syntax, yet, this changes significantly the internal of the change, which did require a not insignificant refactor.

Additionally, as previously mentioned, there were a bunch of changes to goja, including adding class support which also needed to be integrated.

A future breaking change is that using CommonJS style exports along import/export syntax in the same file will no longer be possible.

import http from "k6/http"

exports.default = function() {} // this will start to error out

It will still be possible to import a CommonJS module and require an ES module or use require in an ES module. Having a dependency cycle mixing and matching CommonJS and ESM is also unlikely to work properly, but might do so in particular cases.

This is really expected to have never been done by anyone as there isn't really much of a reason for this, but it is currently supported due to Babel transpiling everything to CommonJS behind the scenes.

Refactoring metrics

The refactoring of metrics is underway, with a PR for more performant internal representation. Unfortunately, this took longer to get to a stable state, and given the size and the type of change, the k6 team decided to hold merging it until very near the end of the cycle. It also doesn't have any noticeable change for most users. Instead, it will be merged early in the v0.41.0 release cycle, and then more changes to use it will be made through the release cycle.

Some of those changes include supporting non-indexable tags, which also have a read PR. This change is also essential for historical reasons connected with how the name tag works. As such, it also needed to be merged to release the internal metric refactor.

Future breaking change: as part of the many changes and experiments, we found out that we can keep url as an indexable tag. Previously the plan was that it along vu and iter will become non-indexable. However, the url tag is heavily used and enabled by default. Because of that (and other internal reasons), the new change will be that url will stay indexable, but if used with name will be overwritten to have the same value as name. Arguably this is what most users would want in the case when they are using name. We plan to add a non indexable raw_url tag for those that do not. As such, we no longer will be printing a warning when url is used in thresholds.

Even so, we did make a bunch of changes to the internals of k6 that will pave the way forward (#2629, #2631).

We are also still working on incorporating the newly developed time series data model for the Prometheus remote-write output extension. We are fixing bugs and improving the extension with the goal of eventually integrating it as a core built-in k6 output module in a future k6 release.

Distributed tracing support itself needs non-indexable tag support. Once that is merged, more work in that direction will be started.

k6 v0.39.0 is here! :tada: It's a small release that includes a bunch of bugfixes and minor enhancements. Much of our focus was on some upcoming big changes. You can read about what's coming up next in the Roadmap and future plans section.

Enhancements and UX improvements

• #2274 and #2560 improved the csv output with support for a new timeFormat option. The possible values are unix (default) and rfc3399. You can also configure it through the K6_CSV_TIME_FORMAT environment variable. Thanks, @rpocklin!
• #2509 added the clear() and delete() methods to the CookieJar object from the k6/http module. Thanks, @Maksimall89!
• #2282 increased the precision of the iteration-progress bar in the UI. Thanks, @m3hm3t and @DarkAEther!
• #2568 added more descriptive error messages when there were problems with parsing a config file.

Bug fixes

• #2523 fixed a gRPC marshaling error when any.proto was used for a type. Thanks, @Flowersea!
• #2534 fixed the return type of Selection.map() from the k6/html module to the correct object types instead of a forced array of strings.
• #2502 made it so k6 waits for scenario executors to fully finish before updating their final progress in the UI, preventing misleading red crosses (#2500).
• #2524 fixed a bug where GoError string contained missing URL values (#2537).
• #2530 fixed a wrong error message on remote resolution.
• #2542 fixed a bug where Rate metric and sub-metric values were shown as NaN in the end-of-test summary if there were no measured values for them during the test run.
• #2558 fixed a panic when trying to set the value of a vu.tags element from k6/execution to null or undefined.
• #2567 fixed a panic when trying to access some k6/execution properties outside of a VU context, e.g. trying to access execution.scenario in setup().

Maintenance and internal improvements

• #2550 updated the used Go version to 1.18.
• #2524, #2551, #2552, #2553, #2554, #2555 updated various Go dependencies in k6.
• #2583 added a deprecation warning for thresholds that use the url, error, vu or iter tags, which will become un-indexable in the future.

Roadmap and future plans

As the lack of big changes in this release suggests, we've focused the last few months' efforts on a few areas that haven't yet been merged into the core of k6.

In this section, we'd like to inform the community about important features that we're currently working on - our short-term roadmap in a sense. We'll also use it to give notice of breaking changes we plan to make in the near future.

k6/experimental/* JS modules

Over the last several k6 releases, among a lot of other refactorings, we've added support for JavaScript event loops (#2228) in k6 VUs and added a new Go API for exposing built-in and xk6 extension modules to user scripts (announcement, docs). This has given us (and any xk6-extension authors!) the ability to better support various asynchronous streaming/messaging/etc. protocols (#882).

We've started building some of these newly possible APIs as xk6 extensions first, to be able to iterate on them more quickly and get some user feedback while we are building them. xk6-websockets, xk6-timers and xk6-redis are some of the first such APIs, but we plan to also work on support for gRPC streaming (#2020), messaging protocols (#1269), a new and better HTTP API (#2461) and many others in the future!

We want to eventually include a lot of these APIs in the k6 core as built-in modules that users can directly use, without needing to mess with xk6 or Go compilation. However, because we try to keep the built-in k6 APIs stable and backwards-compatible, we want to get more user feedback before we do that, while we are still free to iterate and make (hopefully minor) breaking changes.

So, we decided to create a new middle ground between the unstable and purely external xk6 extensions and the stable built-in k6 APIs―built-in k6/experimental/* modules! Our goal is that, starting with the next k6 v0.40.0 release, we'll start releasing some or all of these core-bound extensions as built-in k6 modules under these k6/experimental/ import paths. This will let k6 users, both OSS and Cloud, to give us feedback and help us improve the APIs before we stabilize them.

As is hopefully clear from the name, our usual guarantees of API stability won't apply to these modules while they are still experimental. We reserve the right to make breaking changes in experimental modules, up to and including completely dropping them. We don't expect big breaking changes will need to happen often, but we want to be clear they aren't impossible. Finally, when an API has been stabilized and made available under a regular import path, we'll deprecate its experimental import path. To make the transition easier, both import paths will be available simultaneously for at least one k6 version.

Native support for ECMAScript modules

At the moment, k6 has support for ECMAScript modules (ESM, i.e. import, export, etc.) via automatic transpilation of scripts by the built-in Babel.js. That mostly works, but it has caused some performance and compatibility problems (#824 and #2168 among others), so we want to support ESM modules and all other ES6 features directly in k6, without the need for Babel.js (#2296). goja, the JavaScript runtime we use to evaluate k6 scripts, doesn't yet have native ESM support, so we are currently working on adding it there, to then be able to support ECMAScript modules natively in k6!

That work has been ongoing for a while and we're making progress, but it will likely not be ready in time for the next k6 v0.40.0 release. We are mentioning it here because we will probably need to make a few minor breaking changes and fixes of currently undefined behavior in k6 to support ESM modules natively.

For example, at the moment, some values like the consolidated script options were unintentionally made available globally, in all JS modules of a test, instead of just in the exported options value from the main JS module. That is not the intended or documented behavior, it's somewhere between a bug and undefined behavior, and we'll need to fix it (#2571) and other similar issues like it, starting in k6 v0.40.0. We don't expect many scripts to break because of these fixes, but we'll nevertheless announce them in the release notes of the k6 version that they happen in.

Refactoring metrics

Over the last several k6 releases, we've also slowly been refactoring and improving the metrics internals in k6 (see #2071, #2330, #2426, #2463, #2433, #2442, among others). This has already brought many side benefits and minor bugfixes, and we still have a lot of work left (e.g. #1889, #572, #2430, #1831), but we've finally reached the point where we can almost start implementing some major new features effectively!

One of the upcoming next big steps is the introduction of a "time series" concept internally in k6 (#2580). We'll start to efficiently group samples (i.e. metric measurements) with the same metric and tags into a single TimeSeries, which would unlock many of possibilities that were previously too inefficient to implement.

Another upcoming metrics-related change would be the refactoring of metric tags into two distinct types (#2584) - ones that can be used for indexing (so, in the TimeSeries mentioned above, in sub-metric thresholds, in certain outputs that do aggregation based on tags, etc. ) and ones that cannot (e.g. high-cardinality metadata that's often unique for every data point).

Unfortunately, we'll need to make a few minor breaking changes. The current url, error, vu and iter system tags will be made non-indexable by default, to reflect their usually very high cardinality, so they won't be usable in thresholds. Instead of error, the error_code tag should be used there. And instead of url, users should already use the name tag to avoid issues, see the documentation about URL grouping. k6 v0.39.0 still supports thresholds with these tags, it will just print a warning if they are used.

Prometheus remote-write output

Because of these changes, we'll finally be able to refactor and improve the Prometheus remote-write output extension. Our goal is to make it more efficient, polish its rough edges, and merge it into the k6 core as a built-in output module in the near future.

Tracing support

Another new feature these changes would unlock is support for distributed tracing (#2128).

Because of the upcoming support for high-cardinality unindexable metric tags, we'll be able to safely attach things like the unique trace and span IDs to every metric measurement without negatively affecting other parts of k6.

k6 v0.38.3 is a patch release containing a single fix

Threshold over already defined sub-metrics will result in an error (#2538)

There was a bug where we were checking if a submetric had already been added. Unfortunately, we didn't check that this will work with the one submetric we have by default http_req_duration{expected_response:true}. After v0.38.0 defining a threshold on it would result in an error.

As this definitely shouldn't happen in that case and we don't see a particular case where that will be problematic - adding a submetric again just reuses the already added one instead.

This issue has been addressed in #2539, and k6 v0.38.3 will now lead you add a threshold on http_req_duration{expected_response:true}.

k6 v0.38.2 is a patch release containing a couple of bugfixes!

Threshold over sub-metrics without samples would result in NaN (#2520)

There was a bug in thresholds applied to sub-metrics set to abortOnFail: leading k6 to evaluate thresholds that would have likely aborted before they had a chance of passing (because no samples for the given metric were recorded yet). This bug would have led to such thresholds' results value to be NaN rather than a numerical value. The following script, for instance:

import { check, sleep } from 'k6';

import http from 'k6/http';

export const options = {
  scenarios: {
    iWillFail: {
      exec: 'iWillFail',
      executor: 'constant-vus',
      startTime: '2s',
      vus: 1,
      duration: '30s',
    },
  },

  thresholds: {
    'checks{type:read}': [{ threshold: 'rate>0.9', abortOnFail: true }],
  },
};

export function iWillFail() {
  let res = http.get(`https://test-api.k6.io/`);

  check(res, {
    'read status is 200': (r) => r.status === 200,
  }, { type: 'read' });

  sleep(1);
}

Would result in the following:

✗ { type:read }...: NaN% ✓ 0 ✗ 0  
vus...............: 0 min=0 max=0
vus_max...........: 1 min=1 max=1

This issue was introduced by recent changes to how we handle thresholds in the k6 engine and is now addressed in v0.38.2.

Sub-metrics without values rendered below an incorrect parent metric (#2518)

There was in how thresholds over sub-metrics that didn't receive any samples would be displayed under an incorrect parent metric. For instance, the following script:

import { Counter } from 'k6/metrics';

const counter1 = new Counter("one");
const counter2 = new Counter("two");

export const options = {
    thresholds: {
        'one{tag:xyz}': [],
    },
};

export default function() {
    console.log('not submitting metric1');
    counter2.add(42);
}

Would have led to the following output, where the {tag:xyz} sub-metric is displayed under iterations instead of one:

data_received........: 0 B 0 B/s
data_sent............: 0 B 0 B/s
iteration_duration...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
iterations...........: 1 499.950005/s
  { tag:xyz }........: 0 0/s
two..................: 42 20997.90021/s

When we would have expected it to produce:

one..................: 0 0/s
  { tag:xyz }........: 0 0/s
two..................: 42

This issue has been addressed in #2519, and k6 v0.38.2 now displays sub-metrics under their actual parents, even when they have received no samples.

Special thanks to @efdknittlfrank, who reported and helped us track down the issue.

k6 v0.38.1 is a patch release containing a bugfix!

Threshold sub-metric selectors containing reserved symbols would fail (#2512)

There was a bug in threshold sub-metric selector parsing, which led to errors when users would use specific symbols such as {, } or : as part of their definitions. For instance, thresholds used for sub-metrics with URL Grouping like http_req_duration{name:"http://example.com/${}"} would have led to failures in v0.38.0.

The error messages for invalid metric, sub-metric and threshold definitions were also improved.

Special thanks to @efdknittlfrank, who reported and helped us track down the issue.

k6 v0.38.0 is here! 🎉

New Features!

AWS JSLib

There's a new addition to the officially supported k6 JavaScript libraries: k6-jslib-aws. This library lets users interact with a selection of AWS services directly from their scripts. The library currently implements support for the S3 and the Secrets Manager services.

The AWS JS lib documentation has examples and details on how to use the library in your scripts.

Accessing the consolidated and derived options from the default function (#2493)

The k6/execution core module now lets you access the consolidated and derived options that k6 computed before it started executing the test. You can access consolidated options through the exec.test.options property. Note that consolidated options are frozen and cannot be modified. The k6 execution module's documentation has examples and details on how to use the functionality in your scripts.

import exec from "k6/execution";

export const options = {
    vus: 10,
    duration: "30s",
};

export default function () {
    console.log(exec.test.options.scenarios.default.vus); // 10
}

Tagging metric values with the current scenario stage

With the new consolidated script options, we've added a few helper functions to the k6-jslib-utils library. You can use them to automatically tag all the emitted metric samples by k6 with the currently running stage.

The k6 documentation has examples and details on how to use it.

Dumping SSL keys to an NSS formatted key log file (#2487)

This release adds the ability to dump SSL keys while making TLS connections. You then can use these keys to decrypt all traffic that k6 generates.

To accomplish this, set the SSLKEYLOGFILE environment variable to some file path and run k6. This will populate the file with the keys. Then you can use Wireshark to capture the traffic, decrypt it, and use that for debugging.

Here's an example that uses curl to inspect TLS traffic.

Breaking Changes

console methods now pretty print objects and arrays (2375)

For convenience, all console methods such as console.log() and console.info() will now automatically JSON.stringify() objects and arrays passed to them. Thus, instead of console.log({'foo': 'bar'}) printing [object Object], it will now print {'foo': 'bar'}, which will make the experience of debugging k6 scripts easier and more intuitive.

To achieve the previous behavior, cast the Object to a String, as in console.log(String({'foo': 'bar'})).

export default function () {
    console.log([1, 2, "test", ["foo", "bar"], { user: "Bob" }]);
    // before: 1,2,test,foo,bar,[object Object]
    // after: [1,2,"test",["foo","bar"],{"user":"Bob"}]
}

The Go types in the stats package were moved to the metrics package #2433

For convenience and to facilitate further developments, the types and functionalities that used to live in k6's stats package have been moved to the metrics package. The stats package is, as of v0.38.0, removed in favor of the metrics package. Besides, #2442 removed the stats.New function in favor of initializing new metric via a register.NewMetric call instead.

Deprecation

• #2499 removed support for the deprecated maxVUs option. It had been removed in k6 v0.27.0, however using the CLI flag resulted only in a deprecation warning. Now, using this flag will generate an error.
• This release drops some leftovers from the previous version of our JS module Go APIs. As of v0.38.0, these are now unsupported:
  • The deprecated common.Bind (#2488) and common.BindToGlobal (#2451) functions.
  • The context-based (common/context.go #2488) utils have also been removed.

Enhancements and UX improvements

Stricter thresholds' evaluation before the execution starts (#2330)

k6 v0.37.0 already improved threshold parsing by switching its underlying implementation from JavaScript to Go. k6 v0.38.0 introduces two additional improvements:

• k6 will now parse and evaluate thresholds before the execution starts. If a threshold is invalid, as described below, k6 will immediately exit without starting the load test.
• k6 will now detect invalid thresholds:

  export const options = {
      // ...
      thresholds: {
          // Incorrect thresholds expressions:
          http_req_failed: ["rave<0.01"], // e.g. "rave" is not a valid aggregation method
          // Thresholds applying to a non-existing metrics:
          iDoNotExist: ["p(95)<200"], // e.g. the metric 'iDoNotExist' does not exist
          // Thresholds applying an aggregation method that's unsupported by the metric they apply to:
          my_counter: ["p(95)<200"], // Counter metrics do not support the p() aggregation method
      },
  };
  

Disabling colors (#2410)

In addition to the --no-color CLI flag, the ANSI color escape codes emitted by k6 can now also be disabled by setting the NO_COLOR or K6_NO_COLOR environment variables, following the NO_COLOR standard.

# No color output
K6_NO_COLOR=true k6 run script.js

# No color output
NO_COLOR= k6 run script.js

Support for encrypted TLS private keys (#2488)

You can now use passphrase-protected private keys when authenticating with TLS. Using the password property of an options' tlsAuth object, you can now indicate the passphrase to decrypt a private key. Note that this support is limited to the scope of RFC1423 and does not support PKCS8 keys, as they're not yet supported by the Golang standard library.

export const options = {
    tlsAuth: [
        {
            domains: ["example.com"],
            cert: open("mycert.pem"),
            key: open("mycert-key.pem"),
            password: "mycert-passphrase",
        },
    ],
};

Thanks, @Gabrielopesantos, for the contribution.

Improve JSON output's performance (#2436)

The JSON output was optimized and now should be around 2x more performant at outputting metrics. This means that it either can export twice as many metrics, or use half the resources to do the same amount of metrics.

As a side effect, there is a slight breaking change: the tags field is no longer sorted.

Treat panics as interrupt errors (#2453)

We changed the behavior of how k6 treats Go panics, which may happen because of bugs in k6 or in a JavaScript k6 extension. Previously, the behavior was to catch the panic and log it as an error.

Starting with v0.38.0, whenever k6 observes a Go panic, it logs an error like before, but more importantly, it will abort the script execution and k6 will exit with a non-0 exit code. This will help extension authors to identify issues in their extensions more easily.

Miscellaneous

• #2411 The k6 command-line UI (logo, test description, and progress bars) can now effectively be disabled using the --quiet flag.
• #2429 lib/types now exposes the source of the NullHostnameTrie to simplify access to an original list of the hostnames.

Extensions

PoC for a new Web Sockets JS API

We built a new xk6 extension, https://github.com/grafana/xk6-websockets, with a proof of concept implementation for a new JavaScript Web Sockets API. This API uses the global event loops introduced in k6 v0.37.0 to allow a single VU to have multiple concurrent web socket connections open simultaneously, greatly reducing the resources needed for large tests. It also is a step towards supporting the official Web Sockets JS standard, potentially allowing the usage of more third-party JS libraries in the future.

Please share any feedback you have about the new extension since it's likely that we'll adopt a future version of it into the k6 core in one of the next several k6 releases.

gRPC module refactored to enable gRPC extensions to use it

#2484 moved out in a new dedicated Go lib/netext/grpcext package all the parts not strictly required from the js/k6/net/grpc module for binding the gRPC operations and the JavaScript runtime. It facilitates the development of extensions based on gRPC without the direct dependency on the goja runtime. Furthermore, the new Dial function accepts a grpc.DialOption variadic for customizing the dialing operation.

Event loop testing

With this release, you can export the event loop added in v0.37.0. This lets extension developers test event-loop-dependent APIs.

There were also updates to modulestest.VU to support the new API. Head to GitHub to see it in action.

Bugs Fixed!

• #2456: Fixed a very unlikely panic involving arrival rate executors and execution segments with very low VU counts.
• #2349: Thanks to @rainingmaster it is now possible to leave the options' tlsAuth domains property empty.
• #1346: Thresholds over custom metrics for which no data was collected will now be evaluated.
• #2390: Thresholds over sub-metrics with empty tag values will now return the appropriate results.
• #2480: Fixed an error occurring when passing the data property of an http.file() result as a request body.

Known issues

• @hecnavsanz reported to us that in certain scenarios, k6's UI might inaccurately report a test as failed :x: when it actually succeeded :white_check_mark:. This is caused by some concurrency issue in our codebase that only affects the report's UI, and has no impact on the actual test result. A fix is in the works, but won't be ready for this version. We expect to ship it with the next k6 release instead.

Maintenance

Dependencies

• #2479 updated k6's Goja version to 9037c2b61cbf.
• #2449, #2446, #2444, and #2443 updated k6 Go dependencies to their latest compatible versions.

Other

• #2504 updated our installation instructions to reflect the recent changes in the behavior of the go install command. Thanks, @JamieEdge, for your contribution!
• #2437 refreshed our contribution guidelines.
• #2431 refreshed out our dependencies update workflow, process, and guidelines.

k6 v0.37.0 is here! 🎉 Mainly it contains fixes and ongoing efforts with refactoring.

New Features!

Added experimental basic event loop (#882)

We added basic event loop support in k6 (#2228 and #2373) :tada: This was just the first step and isn't used by any of the existing k6 JS APIs yet. For now, it is only available to xk6 extensions like this one that adds support for setTimeout(), setInterval(), clearTimeout() and clearInterval().

Expect to see more changes related to event loops in the next k6 releases, where event loops will start being used by some core k6 modules! For example, by improving some existing JavaScript APIs to have support for callbacks or return Promise values, so they can be used asynchronously. We expect this change will unlock a lot of previously difficult use cases (see #882), though we'll likely iterate on these new APIs as experimental extensions for a while, to stabilize them before we merge them into the core.

:information_source: If you are an extension developer, please use it and give your feedback. But take into consideration that it's likely that the current Go API may change.

Added an option to output k6 logs to a file through --log-output (#2285)

This is on top of the already supported options for sending logs to stdout/stderr and to Grafana Loki. This new feature speaks for itself with simple usage examples:

k6 run --log-output file=./k6.log --logformat json ./examples/stages.js

And one more with a defined minimal log level:

k6 run --log-output file=./k6.log,level=info --logformat json ./examples/stages.js

Thanks, @alyakimenko for the contribution!

Docs: Using file output

Breaking changes

Introduced stricter thresholds parsing (#2400)

In the past, thresholds were evaluated using a JavaScript runtime. For a multitude of reasons, this wasn't satisfying. As of v0.37.0, thresholds are now parsed directly in Go. As a result, k6 will now return an error message on thresholds that do not strictly match the documented specification, instead of just silently ignoring them. Another change is that when a non syntactically correct threshold expression is detected, k6 will immediately interrupt its execution before even starting the load test run.

Below you can find examples of the thresholds expressions that won't work anymore:

export const options = {
    thresholds: {
        "http_req_duration": [
            // although the aggregation method and values are correct, 
            // the equal sign is invalid; use == or ===
            "rate=200",
            // thresholds do not support javascript expressions anymore
            "throw new Error('wat')",
            // it fails, as foo is not a valid threshold expression's aggregation method keyword
            "foo>500",
        ],
    },
};

Extensions

v0.37.0 finalizes (#2376) the switching of our internal modules (gRPC module refactoring) to a new Go/JavaScript module API.

:warning: It's important to highlight that the old API (e.g. methods like context.WithRuntime, common.Bind and others #2384) is deprecated and will be removed in the next k6 release (v0.38.0). For this release, every extension that isn't using the new API will get a warning message like this:

WARN[0000] Module 'k6/x/sql' is using deprecated APIs that will be removed in k6 v0.38.0, for more details on how to update it see https://k6.io/docs/extensions/guides/create-an-extension/#advanced-javascript-extension

We did migrations for some xk6 extensions (see connected issues to the task #2344). The pull requests can serve as examples on how to transition your extension to the new API.

Docker Repository

We migrated our Docker Hub repository from loadimpact/k6 to grafana/k6 (#2377).

docker run -i grafana/k6 run - <script.js

We will continue publishing our docker image releases as both loadimpact/k6 and grafana/k6 for several more releases, but if you use the old one in your local or CI environments, please plan the migration.

Enhancements and UX improvements

• We continued work on the source map feature: add samples (#2339) and improve loading and parsing (#2355).
• Updated installation from source instructions (#2359). Thanks, @karitham, for making the change.
• Enabled more TC39 tests: optional-chaining (#2338) and async (#2396).
• We updated our Code of Conduct and a few GitHub URLs (#2416).

Bugs fixed!

• http.head started taking a body as a second argument (#2402) by mistake in v0.36.0. It is now back to its old signature http.head(url, [params]).
• Fixed options.scenarios' JSON marshaling (#2392).
• Metrics' names display again in k6's Rest API (#2421).
• We improved argument validation for check(#2387), http.batch(#2415) and metrics' constructors (#2427).

Internals

• We updated our CI to improve developer experience. Dependency and linter checks now run only for pull requests (#2403).
• This release also contains a few refactoring PRs that fix linters errors (#2334, #2331 and #2341), remove global variable usage (#2336, #2358, #2353 and #2357) and remove an unnecessary dependency (#2313) which makes our codebase more consistent and maintainable.
• The headers parameter in k6's GRPC module is marked as deprecated (#2370).
• Switched envconfig to our own fork (#2337) in order to abstract the os package and improve testability.

k6 v0.36.0 is here! 🎉 It introduces a couple of new features which enhance its usability, includes a number of fixes, and the result of ongoing refactoring efforts.

New Features!

Source Maps support (#2082)

Following #2082, k6 now has support for Source Maps. k6 will try to load source maps either from the file system(s) or from inside the script, based on the standard(-ish) //#sourceMappingURL= comments. Furthermore, as k6 internally uses Babel to transform ES6+ scripts to ES5.1+, it will now make use of its ability to generate source maps, including combining with previously generated ones, to report correct line numbers. This should fix #1804; however, we anticipate some more issues will arise, and further tweaking will be necessary.

Thus, given an imported.js module such as:

export function f1() {
  throw "line 2";
}

  throw "line 6";
}

export function f3() {
  throw "line 10";
}

and a k6 test script importing it as such:

import { f2 } from "./imported.js"

export default function() {
  f2();
}

Previous versions of k6 would report an error stack trace indicating an invalid line number in imported.js (10):

ERRO[0000] line 6
        at f2 (file:///some/path/imported.js:10:61(2))
        at file:///some/path/sourcemap.js:4:20(4) executor=per-vu-iterations scenario=default source=stacktrace

Starting with v0.36.0 and source maps support, k6 would now report the exception at the correct line in imported.js:

ERRO[0000] line 6
        at f2 (file:///some/path/imported.js:6:2(2))
        at file:///some/path/loadtest.js:4:2(4)
        at native executor=per-vu-iterations scenario=default source=stacktrace

Temporary warning

Note that if a file size is greater than 250kb and the internal Babel is needed, Babel will not generate source map. This is because during internal testing it was found this takes 3x to 4x more memory, potentially leading to OOM (standing for "Out Of Memory", a state in which the OS kills a process for using too much memory) on bigger inputs. If required, you can control the accepted file size limit via the temporary K6_DEBUG_SOURCEMAP_FILESIZE_LIMIT=524288 environment variable; which will be removed after we no longer rely on Babel (#2296). A pre-generated source map will always be loaded. For more details, check #2345.

Ability to abort tests (#2093)

Thanks to the contribution of @gernest (#2093), k6 now has the ability to abort a test run from within the test script. The newly added test.abort() function in the k6/execution module allows k6 scripts to immediately abort the test execution - the VU that called it will abort immediately and any other VUs in the same or other instances (in the case of k6 cloud) will also be interrupted and abort soon after. Local k6 run tests will exit with a code of 108, so this event can also be easily detected in a CI script.

Aborting is possible during initialization:

import exec from "k6/execution";
exec.test.abort();

As well as inside the default function:

import exec from "k6/execution";

export default function() {
  // Note that you can abort with a specific message too
  exec.test.abort("this is the reason");
}

export function teardown() {
  console.log("teardown will still be called after test.abort()");
}

k6 inspect extended output (#2279)

Following #2279, the k6 inspect command now supports an --execution-requirements flag. When used, the command's output will include fields related to the execution requirements, by deriving k6's configuration from the execution context, and including the maxVUs and totalDuration fields in the output.

Forcing HTTP/1 protocol (#2222)

Thanks to the work of @sjordhani22, #2222 made it possible to force k6 to use version 1.1 of the protocol when firing HTTP requests.

It can be done by setting the http2client=0 value in the GODEBUG environment variable:

GODEBUG=http2client=0 k6 run testscript.js

N.B: the usage of the GODEBUG variable is considered temporary, and expected to change in the future. If you start using this feature, keep an eye out for potential future changes.

Extensions

v0.36.0 marks the switch of some of our internal modules to a new Go/JavaScript module API. We expect this change to make the process of developing internal JavaScript modules and advanced JavaScript extensions easier and more streamlined in the future. Although this switch to a new API does not introduce breaking changes for existing extensions yet, we anticipate deprecating the old extension API (e.g. common.Bind(), lib.WithState(), etc.) at an undecided point in the future.

For more details, see: #2243, #2241, #2239, #2242, #2226, and #2232.

Breaking changes

Restricting file opening to init context

VUs are now restricted to only open() files that were also opened in the init context of the first VU - the one that was initialized to get the exported options from the JS script (__VU==0). While it was somewhat possible to open files only in other VUs (e.g __VU==2) in the past, it was unreliable. #2314 ensures that k6 would now throw an error in a similar scenario. This means that you can still open files only for some VUs, but you need to have opened all of those files in the initial VU (__VU==0).

let file;

if (__VU == 0) {
  open("./file1.bin")
  open("./file2.bin")
} else if (__VU % 2 == 0) {
  file = open("./file1.bin")
} else {
  file = open("./file2.bin")
}

export default () => {
  // use file for something
}

Bugs Fixed!

• We addressed an issue uncovered by our community, which kept our users from using GRPC with multiple services definition in a single proto file. This issue was solved in #2265.
• Thanks to the contribution of @Resousse, we've now updated k6's go-ntlmssp dependency. The updating PR #2290 indeed fixes issues with NTLM Authentication backends returning two authorization headers.

Maintenance

• We have refactored our implementation of the RampingVU executor, for better clarity and maintainability. See #2155.
• #2316 relaxed quite a few of the code linting rules we applied to k6's code. It also revamped our Makefile, so the new make ci-like-lint target will run the exact same golangci-lint version that will be used in our GitHub Actions CI pipeline.
• #2304 prepared the removal of external dependencies from k6's JSONAPI compliant REST API, and deprecated the api.v1's client.Call method in favor of its newer client.CallAPI counterpart. It allows us to both reduce our reliance on external dependencies and improve its maintainability.
• We have updated our Goja dependency, our JS interpreter, to its latest available version. Unfortunately, some of the new features are not always usable, yet. Namely, Goja now supports the optional chaining syntax, but the Babel version we use presently does not. Which means that if Babel needs to be used, optional chaining can't be. See #2317 and #2238.
• Thanks to @knittl, #2312 upgraded loadimpact/k6 docker image base to Alpine 3.15.

Known Bugs

• #2226 introduced an unintended breaking change to http.head(). The signature in k6 v0.35.0 was http.head(url, [params]) and was inadvertently changed to http.head(url, [body], [params]) in v0.36.0. That change will be reverted in k6 v0.37.0, but until then, we suggest users use the stable http.request('HEAD', url, null, params) API for HTTP HEAD requests that need to specify custom parameters. Thanks, @grantyoung, for reporting the problem (#2401)!

k6 v0.35.0 is here! :tada: It introduces several new features that nicely enhance its usability and also contains a whole lot of fixes and ongoing efforts with refactoring.

In total, we have closed 14 issues. We have also branched out three new xk6 extensions during this release :star:

New features

Ability to set VU-wide custom metric tags (#2172)

k6 now supports setting tags for VUs as part of the Execution API with an easy key-value interface. These tags are attached to the metrics emitted by the VU. Example usage:

import http from 'k6/http';
import exec from 'k6/execution';

export const options = {
    duration: '10s',
    vus: 3,
};

export default function () {
    exec.vu.tags['mytag'] = 'value';
    exec.vu.tags['vuId'] = exec.vu.idInTest;

    console.log(`mytag is ${exec.vu.tags['mytag']} and my VU's ID in tags ${exec.vu.tags['vuId']}`);

    // the metrics these HTTP requests emit will get tagged with `mytag` and `vuId`:
    http.batch(['https://test.k6.io', 'https://test-api.k6.io']);
}

One of the most requested use cases for this feature is that now we can tag all metrics with the current stage number. With a bit of JS code it is possible to calculate which stage of a ramping-vus or ramping-arrival-rate scenario the VU is currently in. This in turn allows the setting of thresholds only on the metrics that were emitted in specific stages of the test run! :tada:

There are some caveats, however: values can be only of String, Number or Boolean type, while values of other types will result either in a warning or an exception if throw option is enabled. Additionally, given that k6 has a whole bunch of system tags, one should be careful with using them as keys. You can read complete information about VU tags in k6/execution docs.

Initial basic support for JS promises

With the goja update in #2197, you can now make a Promise and chain it in your k6 scripts:

export default function () {
    var p = new Promise((resolve, reject) => {
        console.log('do something promising!');
        reject('here');
    });

    p.then(
        (s) => { console.log('fulfilled with', s) },
        (s) => { console.log('rejected with', s) },
    );
}

It must be noted that Promises are not used by k6 itself yet but this addition is a stepping stone for implementing async functionality in future releases. Thanks, @dop251, for your awesome work in developing goja! :heart:

Support for gRPC server reflection (#2160)

k6's gRPC capabilities were extended with a support for server reflection which allows one to use gRPC even without a proto file at hand. In other words, the following script is now possible:

import grpc from 'k6/net/grpc';
import { check } from "k6";

let client = new grpc.Client();

export default () => {
	client.connect("127.0.0.1:10000", {plaintext: true, reflect: true})
	const response = client.invoke("main.RouteGuide/GetFeature", {
		latitude: 410248224,
		longitude: -747127767
	})

	check(response, {"status is OK": (r) => r && r.status === grpc.StatusOK});
	console.log(JSON.stringify(response.message))

	client.close()
}

You can read more about the protocol here. Thanks, @joshcarp, for putting a lot of effort into this feature!

Other changes and UX improvements

• Support for cookie jars in k6/ws (#2193).
• Forbid metric.Add calls to let NaN values through. Instead, k6 will log nice warnings or throw an exception if --throw is enabled (#1876, #2219).
• Support for compression in websockets (#2162). Thanks, @cooliscool!
• Switch to camel case for CLI options to the outputs (#2150). Thanks, @JosephWoodward!
• Much neater error message on nil response body (#2195). Thanks, @daniel-shuy!

New xk6 extensions

xk6-browser

xk6-browser is a browser automation extension which relies on Chrome Devtools Protocol. With xk6-browser, you can interact with the browser to test your web applications end-to-end while accessing all of the k6 core features, including protocol-level APIs and other k6 extensions. It’s a single tool for both protocol and browser-level testing.

The browser extension comes with an API that aims for rough compatibility with the Playwright API for NodeJS, meaning k6 users do not have to learn an entirely new API.

xk6-output-remote-write

Prometheus is now officially supported in k6 OSS with a xk6-output-remote-write extension. This is an output extension with implementation for Prometheus Remote-Write protocol which means that beyond Prometheus, any compatible remote-write solution can be used with it. You can read the full guide to using the extension in the relevant tutorial.

xk6-output-influxdb

After hard work at working out how to integrate InfluxDB v2 API, it was decided to pull that integration into a new xk6-output-influxdb extension for now. The built-in influxdb output in k6 still supports only InfluxDB v1, as before, with some minor optimizations (#2190).

Please try out the new extensions and tell us what you think!

Breaking changes

• The addition of common metrics registry (#2071) no longer allows defining custom metrics with the same name as one of the builtin metrics, e.g. new Counter("http_req_duration") will now abort. Similarly, an attempt to redefine a metric with the same name but with different type will error out. Builtin metrics may no longer be referenced as global objects in xk6 extensions either.
• Fix inconsistency in environment variables' names: use K6_NO_SETUP and K6_NO_TEARDOWN options instead of NO_SETUP and NO_TEARDOWN (#2140).
• Module interfaces were changed as part of refactoring efforts. Any JS module that needs access to the VU must now implement the new interfaces. This change can impact some xk6 extensions (#2234).

Bugs fixed!

• Fix of a misleading sorting of custom submetrics in the default end-of-test summary (#2198). Thanks, @knittl!
• Fix for extensions depending on afero.FS: implement a newer version of the afero.FS interface for internal filesystems so that extension depending on that or newer version can be built (#2216).
• Fix for websockets: websockets now use the global User-Agent setting (#2151). Thanks, @cooliscool!
• Fixes for tests, Github actions, and Loki integration (#2205, #2153, #2220).

Maintenance

• Update a whole bunch of dependencies (#2159, #2170, #2165).
• Switch to Go 1.17 (#2156). Thanks, @b5710546232!
• Get rid of mapstructure (#2223).
• Minor but necessary cleanups (#2164, #2192).

hacktoberfest

k6 participated in this year's hacktoberfest and we would like to thank all contributors! Here're some additional improvements made by the community members:

• Add multi-message WebSockets tests (#2184).
• Try out the new and shiny Github forms which are already improving the formatting of k6's new issues (#2174,#2179).
• An improved writing style and correctness in our README (#2189, #2152, #2169, #2181) and in some other places (#2182).

Thank you, @knittl, @cooliscool, @JosephWoodward, @b5710546232, @nontw, @divshacker, @daniel-shuy, @Sayanta66, @marooncoder09, @idivyanshbansal, @saintmalik, @EricSmekens, for helping make k6 better :smile:

k6 v0.34.1 is a patch release with a few minor bugfixes:

• There was a minor bug in the new k6/execution API added to k6 v0.34.0 - some of its properties weren't usable with the externally-controlled executor (#2132).
• goja, the JavaScript runtime that k6 uses, was updated to its latest version (#2135), which fixed a couple of bugs:
  • A newly introduced JS bug from k6 v0.34.0, where rest parameters were undefined when the functions also had an internal lambda (#2131). Thanks for reporting, @efdknittlfrank!
  • An old JS bug, first introduced in k6 v0.28.0, which caused Response.json() to not have a length property when the response was a JSON array, i.e.response.json().hasOwnProperty('length') was returning false (#2133). Thanks for reporting, @julien-sugg!

k6 v0.34.0 is here! :tada: It introduces the already announced k6/execution API and includes some more enhancements and a bunch of minor bug fixes.

New k6 JavaScript API - k6/execution

The k6/execution module allows the fetching of information about the current VU, instance (mostly relevant in distributed/cloud) or scenario execution state through a series of exported properties.

How to use

Some of the previously suggested solutions with the globally available __VU and __ITER values, such as for getting a unique object per iteration from an array or SharedArray, can be done by using the scenario.iterationInTest property, which is guaranteed to be unique across VUs, even for distributed or cloud tests. For example:

import exec from "k6/execution";
import { SharedArray } from "k6/data";

const data = new SharedArray("my dataset", function(){
  return JSON.parse(open('my-large-dataset.json'));
})

export const options = {
  scenarios :{
    "use-all-the-data": {
      executor: "shared-iterations",
      vus: 100,
      iterations: data.length,
      maxDuration: "1h"
    }
  }
}

export default function() {
  // this is unique even in the cloud
  var item = data[exec.scenario.iterationInTest];
  http.post("https://httpbin.test.k6.io/anything?endpoint=amazing", item)
}

You can read the full documentation here.

Enhancements and UX improvements

• Warn Windows users on importing dependencies or opening files as absolute paths (#2078).
• Pass setup data object into handleSummary callback (#2103). Thanks, @SamuelJohnson01997!

Breaking changes

• The deprecated outputs Datadog and Kafka have been removed (#2081).

Bugs fixed!

• Use the POST HTTP request method instead of GET for pushing logs to Loki (#2100).
• Encode the blacklistIPs option using the CIDR notation in JSON (#2083).
• ext.loadimpact option has the same precedence as the script configuration during the consolidation process (#2099).
• The WebSocket connection used for tailing logs from the k6 Cloud is reestablished in the case of an unexpected error (#2090).

Internals

• A simpler and clearer API has been added as an alternative to common.Bind, which also gives JS modules and extensions easy access to some useful internal objects and runtime information (#2108). This API is not yet stable, it's very likely to change more in future k6 versions.
• Speeding ups TC39 tests using a pool of Babel compilers (#1839).
• Goja and some internal dependencies have been updated adding the native support for Arrow functions, Destructuring, Default arguments and Computed properties features. For the same reason, the relative Babel's plugins supporting those features are not required anymore so they have been disabled (#2109, #2092).

k6 v0.33.0 is here! :tada: It's a small release that includes a bunch of minor bugfixes and enhancements, but is also laying the groundwork for some major new features like the upcoming k6/execution API in k6 v0.34.0.

Acquired by Grafana Labs

Load Impact, the company behind k6, was acquired by Grafana Labs! :tada: Nothing changes regarding the k6 development for now, and any changes in the future will only be in the direction of accelerating our existing roadmap and plans, as well as better integration between k6 and the awesome Grafana tools. For more details, see the official Grafana press release.

Enhancements and UX improvements

• The --verbose help message and the statsd warning message were improved (#2005). Thanks, @vishalkuo!
• The noColor k6 option and the current UI state are now propagated to the handleSummary() function. The state object has the isStdErrTTY, isStdOutTTY and testRunDurationMs keys (#1975).
• The error message when an HTTP request times out was improved (previously it was context deadline exceeded) and it now has an error_code value of 1050 (#2008). Thanks, @vishalkuo!
• Script errors will no longer have the confusing GoError prefix in their error messages (#1775).
• All custom metric objects now have a name property (#2058 and #2076). Thanks, @olimpias and @david-gourde!
• Top-level JS arrays will now be properly encoded when sent in the body of a application/x-www-form-urlencoded request (#2060). Thanks, @noelzubin!

Bugs fixed!

• The minIterationDuration option was uninterruptible and could delay the stopping of a scenario even after gracefulStop had expired. (#2035).
• The error_code detection for HTTP/2, x509 and TLS (and potentially others) was unreliable (#2025).
• k6 used to panic when responseType was binary, but there was no response body actually returned, e.g. when there was an HTTP error (#2041).
• The throw option was not respected when there was an invalid URL (#2045). Thanks, @gchaincl!
• k6 would return an exit code of 103 instead of 107 for script errors when initializing non-service VUs (#2046).
• Deleted library versions from cdnjs could previously cause a panic (#2047).
• The correct error message for missing files was not shown when the filename contained spaces (#1973).
• The regular expressions for the github and cdnjs "magic" loaders were slightly wrong (#2066).
• A potential (harmless) data race could have been caused by an unintentional copying of a data struct (#2067).
• The segmentation of small ramping-arrival-rate scenarios was not optimal (#1863).

Internals

• The default end-of-test summary is now completely generated by the same k6-summary JS code that is hosted on jslib.k6.io (#1975). That PR also improved the k6 TTY detection and removed a few Go dependencies and code hacks, though it also caused us to bump the minimum required Go version for compiling k6 to Go 1.16 (because of its usage of go:embed).
• Arrival-rate executors will no longer create a new goroutine for every new iteration (#1957 and #2038).
• We have enabled GitHub's CodeQL checks for the Go parts of the repo (#1961). Thanks, @jfcg!
• We have added the necessary k6 core changes for providing execution information to scripts #1863! This was the groundwork for the extended replacement of the __VU and __ITER execution context variables we plan to introduce. The new API will be able to return other information as well, for example which scenario the current iteration is in, what's the number of the current VU/iteration globally across all k6 instances, or in the current scenario, etc. These APIs are still not available to JS scripts, but we plan to expose them via the k6/x/execution xk6 extension for now and iterate on them in the following weeks, releasing a stable version in k6 v0.34.0.

Breaking changes

• The k6 cloud exit code for a failed cloud test was changed from 99 to 97 (#2046).
• The default value of K6_STATSD_TAG_BLOCKLIST and K6_DATADOG_TAG_BLACKLIST is now vu,iter,url (#2063).
• The __ITER execution context variable is no longer set in setup() and teardown() due to #1863. This might be better classified as removing a previously undefined behavior instead of a breaking change, but it's still worth mentioning.

k6 v0.32.0 is here! :tada: It's a smaller release, featuring mostly chores that we never got to or that needed to be done in this release cycle, but also includes a significant performance improvement for arrival-rate executors and some breaking changes.

Move out of Bintray

Bintray has stopped servicing users on 1st of May, which meant that we needed to move the deb and rpm repositories out of there before then. Please follow the new installation instructions and the "Note about Bintray" to find out how to remove the old repository if you have used them.

Notable changes

Move all outputs to new Output interface introduced in v0.31.0 and other related changes

We started on this in the previous v0.31.0 release and now all internal outputs implement the new Output interface we provided for xk6 output extensions. Additionally one of the old built-in outputs is being deprecated and one renamed:

1. The kafka output has been somewhat neglected since it was added 3 years ago. Additionally it brings a lot of complexity and isn't well understood by anyone on the team. Given that output extensions are possible since the last version, the Kafka output has been moved to one. The built-in output will continue to work for a few more k6 versions, emitting a deprecation warning when used, so that everyone has time to transition to the extension. All future improvements will happen only in the extension, the built-in output is frozen until it's dropped.
2. We are also deprecating/renaming the datadog output. It should've probably always been just a configuration of the statsd output and now in k6 v0.32.0, it is going to be just that. We've added a new K6_STATSD_ENABLE_TAGS option to the statsd output, which, when enabled (it's false by default), will send metric tags the same way the datadog output did before. That is, instead of using the datadog output, you should use the statsd one with K6_STATSD_ENABLE_TAGS=true. Additionally, the new K6_STATSD_TAG_BLOCKLIST option can be used to not send tags that the user doesn't want to, similar to the old K6_DATADOG_TAG_BLACKLIST option. This makes it cleaner to also emit metrics to other services that accept the same data and tag formats, such as New Relic, Amazon Cloudwatch, and statsd >v0.9.0. The old datadog output will still work for a few k6 versions, emitting a warning to switch to statsd when used.

Apart from a message about them being deprecated, nothing should be changed from the actual refactoring yet, but we advise users to use the proposed alternatives starting with this release.

json output emits thresholds defined on the metrics (#1886)

Previous to this change thresholds were not included in the json output. Now the Metric JSON object will get its thresholds field properly populated.

Thanks to @codebien for this contribution!

cloud output has an option to abort the test if aborted from the cloud (#1965)

In v0.26.0 we made the cloud output stop emitting metrics if it gets a particular error from the backend, as that meant that the test was aborted in the cloud. Now we added K6_CLOUD_ABORT_ON_ERROR to be able to say that it should not only stop emitting metrics, but also stop the execution of the local k6 test run. The default configuration is false, so it is backwards compatible. This also works when the test is aborted by the user or if cloud execution limits are reached, which would also lead to the test being aborted.

Full stack traces for init context and setup/teardown exceptions (#1971)

For a long time, if there was an exception in either the init context or in the setup() or teardown() invocations, the stack trace would be just the last line, making it really hard to debug issues there. Now there is a full stack trace and, as such errors will result in aborted k6 execution, k6 run will also exit with exit code 107, signalling a script error.

Considerable performance improvements for arrival rate executors (#1955)

Due to a wrong re-usage for a particular internal data structure, the arrival rate executors were having a much worse performance than expected. With this release they should be a lot more performant, especially with large numbers of VUs.

Updating the majority of our dependencies and dropping some

The list is too long and we have been postponing updating for quite some time now, but we have finally updated all our dependencies that we don't want to drop. This could lead to some stability problems, which is why it was done early on in the cycle. While the team has not found any regressions, given all the updates we could have missed something so please open a issue if you find anything.

Some notable updates:

• goja, the JS engine we use got support for let/const which allowed us to disable a Babel plugin. Previous to this if you had a particularly long script sometimes Babel took upwards of 30 minutes to transpile. Now even our worst contender that previously took 51 minutes is transpiled in less than a minute :tada:. Also globalThis is now available.
• updating the gRPC libraries fixed bug (#1928) and probably others. (#1937)

ArrayBuffer is now supported in all JS APIs dealing with binary data, including in WebSocket messages (#1841)

Besides the minor breaking changes (see the "Breaking changes" section below), it's now possible to send binary WS messages with the socket.sendBinary() function and to receive binary messages with the binaryMessage event handler:

const binFile = open('./file.pdf', 'b');

export default function () {
  ws.connect('http://wshost/', function(socket) {
    socket.on('open', function() {
      socket.sendBinary(binFile);
    });

    socket.on('binaryMessage', function(msg) {
      // msg is an ArrayBuffer, so we can wrap it in a typed array directly.
      new Uint8Array(msg);
    });
  });
}

Official arm64 releases for macOS and Linux (#2000)

We will now publish binary arm64 releases for macOS and Linux for new k6 releases. Support for these new architectures should be stable, given Go's cross-platform compatibility, but please report any issues you experience.

Other enhancements and UX improvements

• Options: k6 will now warn you on unrecognised configuration JS options in most cases instead of just silently ignoring them. (#1919)
• error_code: the tag error_code should now be set more accurately in some cases. (#1952)
• TLS: dropped support for SSLv3 encryption. This was dropped by Go, but now we no longer consider ssl3.0 a valid value for the tlsVersion k6 option. Thanks @codebien! (#1897)

Bugs fixed!

• Arrival-rate executors could in some cases report twice as many used VUs than what was actually true. (#1954 fixed by #1955)
• In cases of an error while reading the response body, the newly added responseCallback in v0.31.0 would be evaluated with the returned status code, while the reported one would be 0, as the response errored out and k6 does not return incomplete responses. Now responseCallback will also receive a 0 status. (#1962)
• Fix Kafka output not being usable with the InfluxDB format after v0.31.0 changes. (#1914)
• Error out with a user friendly message if ramping-vus executor would've not run a single iteration instead of just doing nothing. (#1942)

Internals

• JS: Added a way for JS modules to have per VU initialization and object. This can also be used by xk6 modules by implementing go.k6.io/k6/js/modules#HasModuleInstancePerVU. (Part of #1911)

Breaking changes

Support for ArrayBuffer in all k6 JS APIs (#1841)

Continuing from k6 v0.31.0, we're finalizing the transition to ArrayBuffer values for working with binary data. This release introduces some changes that might break scripts that relied on the previous array of integers or string result types, returned by some of our JS APIs. Specifically these cases now return ArrayBuffer instead: open(..., 'b'), HTTP response bodies for requests that specified responseType: 'binary' (including when http.batch() is used), crypto.randomBytes(), hasher.digest('binary') and encoding.b64decode(). The previous default behavior of returning string from encoding.b64decode() can be replicated with a new optional format argument and a value of "s": encoding.b64decode("5bCP6aO85by-Li4=", "url", "s"). Most of these shouldn't cause issues if the script is simply passing the values to another k6 API (e.g. opening a file as binary and passing it to http.post()), but in other cases the script will need to be modified to wrap the ArrayBuffer in a typed array view and use that API instead.

Deprecating official 32-bit binary releases (#2000)

We have stopped offering official builds for 32-bit Windows and Linux binaries, as they were causing issues in some cases, and users were likely using them by mistake instead of necessity.

Moved repo and renamed k6 Go module paths to go.k6.io/k6 (#2010)

We moved the repository location from https://github.com/loadimpact/k6 to https://github.com/k6io/k6. Instead of also moving the Go module paths to the new repo location, we decided to use custom ones with our own domain, for more control. This will be a breaking change for all xk6 extensions, since they import parts of k6 by source to register themselves.

New .deb and .rpm repositories

We already mentioned this in the v0.31.0 release, but because of the Bintray shutdown, we had to move our .deb and .rpm repositories. They are now located at dl.k6.io, and you can use the updated installation instructions to transition to them.

k6 v0.31.1 is a patch release with a single bugfix.

The bugfix is about the cloud output and the new http_req_failed metric in k6 v0.31.0. Due to additional state being used for its transport to the k6 cloud, and a misunderstanding of what a functional call from a library dependency does, the http_req_failed values were always set to 1. This did not affect any other output or the end of test summary. (#1908)

k6 v0.31.0 is here! :tada: It's a smaller release with some significant performance improvements, a new http_req_failed metric and changes to the output subsystem that enable output extensions with xk6!

New features

Output cleanup and extensions (#1874)

The state of k6's output packages has been a development pain point for a long time, which made it difficult to add new outputs in a consistent way. In the refactor done in v0.31.0, this has been mostly addressed and outputs now implement a simpler and cleaner Output interface.

In addition to this, it is now possible to implement custom k6 output extensions in Go with xk6! This is very useful if you use a system that's currently not supported by the built-in outputs, or need some custom handling of the metrics k6 produces.

Writing output extensions is done very similarly to how JS module extensions are currently written, though instead of calling js/modules.Register(), you should implement the new Output interface and call output.RegisterExtension() with your constructor.

We are working on the proper documentation for this, as well as the overdue xk6 documentation about JS extensions, so keep a lookout for those on k6.io/docs.

Marking requests as failed (#1856)

It's now possible to declare expected HTTP response statuses for either the entire test or for individual HTTP requests, and k6 will emit a new http_req_failed metric as well as tag HTTP metrics with expected_response: <bool>. By default, k6 will now fail requests that return HTTP 4xx/5xx response codes.

For example:

import http from 'k6/http';

// Set expected statuses globally for all requests.
http.setResponseCallback(http.expectedStatuses({min: 200, max: 399}, 418));

export default function () {
  // This request will be marked as failed.
  http.get('https://httpbin.test.k6.io/status/400');
  // This request will be considered as "passed" because of the responseCallback override.
  http.get('https://httpbin.test.k6.io/status/400', { responseCallback: http.expectedStatuses(400) });
}

Running this script will produce a summary like:

http_req_duration..............: avg=204.57ms min=203.31ms med=204.57ms max=205.82ms p(90)=205.57ms p(95)=205.7ms
  { expected_response:true }...: avg=203.31ms min=203.31ms med=203.31ms max=203.31ms p(90)=203.31ms p(95)=203.31ms
http_req_failed................: 50.00% ✓ 1   ✗ 1

Note the new http_req_duration sub-metric for expected responses only, and the new http_req_failed Rate metric. This new metric and metric tag have many potential use cases, and one of the most important ones is the ability to set better thresholds. For example:

• 'http_req_failed': ['rate<0.1'], i.e. fail the test if more than 10% of requests fail.
• 'http_req_duration{expected_response:true}': ['p(95)<300', 'p(99.9)<500'] - fail the test if the the 95th percentile HTTP request duration is above 300ms or the 99.9th percentile is above 500ms; specifying expected_response:true here may be important, because a lot of times failed requests may return more quickly than normal ones, thus skewing the results and wrongly satisfying the threshold.

If the response callback is not specified, the default expected statuses will be {min: 200, max: 399}. The previous behavior of not emitting anything can be achieved by setting the callback to null, i.e. http.setResponseCallback(null). Additionally, the expected_response tag can be disabled by removing it from the default list of system tags, e.g. k6 run --system-tags 'proto,subproto,status,method,url,name,group,check,error,error_code,tls_version,scenario,service'.

The http.setResponseCallback() is planned to allow arbitrary JS functions to process responses in the future, but for now only the http.expectedStatuses() callback is supported.

Other enhancements and UX improvements

• JS: Because of the awesome improvements to goja, the JS runtime k6 uses, it's no longer necessary for k6 to load core.js to polyfill missing JS features when using the default --compatibility-mode=extended. So in v0.31.0 core.js has been dropped entirely, yielding some significant CPU and memory usage improvements. The actual numbers will depend on the use case, but for simple tests users can expect a memory drop of about 2MB per VU (from ~2.7MB to ~600KB), and a slight CPU decrease of about 5-10%. For more complex tests with a lot of JS code this benefit won't be as pronounced. Another benefit of this change is that initializing VUs and starting a test is substantially faster than before! (#1824)
• JS: Also because of goja improvements, some unused Babel plugins were disabled which should have minor performance benefits as well. (#1822)
• JS: Expanded ArrayBuffer support in most internal modules, so now you can pass ArrayBuffer to http.file(), in k6/encoding and k6/crypto functions. This makes working with binary files more efficient as it doesn't require string translations. In upcoming versions we plan to expand this to the WebSocket module, as well as make some potentially breaking changes for APIs that currently return an array of integers or string (see the details in the Breaking Changes announcement below). (#1800)
• The Docker image base was updated to Alpine 3.13. Thanks @andriisoldatenko! (#1821)
• The Debian package now includes ca-certificates as a dependency. Thanks @Bablzz! (#1854)

Bugs fixed!

• Execution: Aborting a test during VU initialization (e.g. with ^C) will now properly propagate to any used outputs. (#1869)
• Execution: A race condition between the Engine and the outputs' finalization code was fixed, ensuring that all metrics are properly emitted before exiting. (#1869)
• Execution: Another race condition in the Engine was fixed, which may have resulted in the end-of-test summary missing some of the last test metric data. (#1888)
• Cloud: the test name is now properly validated and will raise an error if not set via the ext.loadimpact.name JS option or config, or the K6_CLOUD_NAME environment variable. (#1870)
• JS: Babel is now also run on compilation errors, which improves support of some obscure language features. (#1861)
• JS: SharedArray introduced in v0.30.0 can now be iterated with forEach. (#1848)

Internals

• JS: SharedArray was rewritten using goja.DynamicArray making it more performant and easier to reason about. (#1848)
• JS: Some TC39 tests for unsupported features were disabled, improving the runtime of the test suite. (#1816)
• CI: Some more tests were enabled on Windows. (#1855)

Breaking changes

• JS: While we don't expect the core.js removal and Babel changes to impact the vast majority of users, those were substantial changes in how k6 interprets JS and a minority of users might experience issues with their tests. Please report any unexpected JavaScript errors by creating a GitHub issue. In particular Promise is now undefined, and some unused Babel plugins like transform-es2015-for-of and transform-regenerator were also removed. This means that some workarounds like the ones mentioned here and here also won't work as is and will need additional polyfills and plugins to work properly.

Planned future breaking changes

The following are not breaking changes in this release, but we'd like to announce them so users can prepare for them in upcoming releases (likely k6 v0.32.0).

• JS: The ArrayBuffer changes in this release are backwards compatible and shouldn't cause any issues, but in v0.32.0 some JS APIs that currently return an array of integers or string for binary data will return ArrayBuffer instead. This is the case for open() when used with the 'b' argument, response bodies for requests that specify responseType: 'binary', crypto.randomBytes(), hasher.digest('binary'), and encoding.b64decode(). Response.json() and Response.html() will also probably stop working when used with requests that specify responseType: 'binary'. These changes shouldn't be a problem for most users that were simply using these values to pass them to other internal modules (e.g. opening a binary file and passing it to http.post()), but if the scripts modified the binary data or depended on the current array of integers or string values they will need to be adapted to use typed arrays instead. You can follow the discussion in PR #1841 and issue #1020.
• As part of the rebranding of Load Impact to k6, the k6 GitHub repository will be moved from https://github.com/loadimpact/k6 to https://github.com/k6io/k6 . Additionally because of Go's usage of URLs in package imports, the URL will be changed throughout the codebase. Since GitHub will maintain a redirect from the old location we don't expect this to impact a lot of users, or even external k6 contributors and xk6 developers, but be aware that the new URL should be used moving forward.
• Because of the sunsetting of Bintray, the DEB, RPM, MSI and Chocolatey package repositories currently hosted on Bintray will be moved to a self-hosted solution sometime in the upcoming weeks. We'll communicate these changes via our blog as well as the official documentation.

k6 v0.30.0 is here! :tada: It was a bit of a slow after-holiday release, but it still packs a few major new features and improvements that users have been requesting for a long time!

New features

Share memory between VUs using read-only arrays (#1739)

k6 has long had an issue with the handling of big data files with test fixtures. For example, if you have a huge users.json file with test users for your application:

[
  {"username": "user1", "password": "password1", "other": "some-long-data-....-1"},
  {"username": "user2", "password": "password2", "other": "some-long-data-....-2"},
  // ... ~1 million more users or more... :D
  {"username": "user999999", "password": "password999999", "other": "some-long-data-....-999999"}
]

If you just use JSON.parse(open('users.json')) in your script, then every VU will have a copy of the whole huge data set. Every VU in k6 is a separate JavaScript runtime, so there wasn't a thread-safe way to share data between them. Until now, that is!

We've added a new built-in SharedArray object in the new k6/data module that allows VUs to share read-only data:

import { SharedArray } from 'k6/data';
import { sleep } from 'k6';
import http from 'k6/http';

let users = new SharedArray('someName', function () {
    // This function will be called only once, in the first init context
    // execution. Every other VU will just get a memory-safe read-only reference
    // to the already loaded data.
    console.log('Loading users.json, this happens only once...');
    // You are not restricted to JSON, you can do anything - parse a CSV or XML
    // file, generate random data, etc. - as long as you return an array.
    return JSON.parse(open('users.json'));
});

export let options = { vus: 10, duration: '30s' };
export default function () {
    let randomUserID = Math.floor(Math.random() * users.length);
    let user = users[randomUserID]; // alternatively, we can also use __VU and/or __ITER
    console.log(`VU ${__VU} is running iteration ${__ITER} with user ${user.username}...`);
    http.post('https://httpbin.test.k6.io/post', JSON.stringify(user));
    sleep(Math.random() * 2); // or, better yet, use arrival-rate
}

Notice how Loading users.json is logged only once, but each VU uses the users variable like a normal JS array. The data is read only once and we have just a single copy of the huge array in memory! Behind the scenes, k6 uses a JS Proxy to transparently copy only the row each VU requests in users[randomUserID] to it. This on-demand copying is a bit inefficient, but it's still leagues better than having a copy of the huge array in every VU! And you can avoid the copying in every iteration by pinning the data used by every VU and having let user = users[randomUserID] in the init context!

And yes, you can have multiple SharedArray objects in the same script, just make sure to give them unique names - this is what someName in the script above was for. Because VUs are independent JS runtimes, we need some way to differentiate between the different shared memory objects, so we require them to have unique names. These names are also the IDs that any xk6 extensions would need to use to access them.

This works both locally and in the cloud. We advise everyone who deals with large data files to wrap them in a SharedArray and give this new feature a try. The required script changes should be minimal, while the memory usage should be significantly lower. Hopefully, we can finally consider one of the biggest blockers k6 users have had for a long time solved! :tada:

Support a handleSummary() callback at the end of the test (#1768)

You can now export a function called handleSummary() and k6 will call it at the end of the test run, after even teardown(). handleSummary() will be called with a JS object containing the same information that is used to generate the end-of-test summary and --summary-export, and allows users to completely customize how the end-of-test summary looks like.

Besides customizing the end-of-test CLI summary (if handleSummary() is exported, k6 will not print the default), you can also transform the summary data to various machine or human-readable formats and save it to files. This allows the creation of JS helper functions that generate JSON, CSV, XML (JUnit/xUnit/etc.), HTML, etc. files from the summary data. Even binary formats like PDF are not out of reach, potentially, with an appropriate JS library that works in k6! You can also send the generated reports to a remote server by making an HTTP request with them (or using any of the other protocols k6 already supports)! Here's a simple example:

import http from 'k6/http';
import k6example from 'https://raw.githubusercontent.com/loadimpact/k6/master/samples/thresholds_readme_example.js';
export default k6example; // use some predefined example to generate some data
export const options = { vus: 5, iterations: 10 };

// These are still very much WIP and untested, but you can use them as is or write your own!
import { jUnit, textSummary } from 'https://jslib.k6.io/k6-summary/0.0.1/index.js';

export function handleSummary(data) {
    console.log('Preparing the end-of-test summary...');

    // Send the results to some remote server or trigger a hook
    let resp = http.post('https://httpbin.test.k6.io/anything', JSON.stringify(data));
    if (resp.status != 200) {
        console.error('Could not send summary, got status ' + resp.status);
    }

    return {
        'stdout': textSummary(data, { indent: ' ', enableColors: true}), // Show the text summary to stdout...
        'junit.xml': jUnit(data), // but also transform it and save it as a JUnit XML...
        'summary.json': JSON.stringify(data), // and a JSON with all the details...
        // And any other JS transformation of the data you can think of,
        // you can write your own JS helpers to transform the summary data however you like!
    }
}

k6 expects handleSummary() to return a {key1: value1, key2: value2, ...} map. The values can be string or ArrayBuffer and represent the generated summary report contents. The keys should be strings and determine where the contents will be displayed or saved: stdout for standard output, stderr for standard error, or a path to a file on the system (which will be overwritten).

The format of the data parameter is similar but not identical to the data format of --summary-export. The format of --summary-export remains unchanged, for backwards compatibility, but the data format for this new k6 feature was made more extensible and had some of the ambiguities and issues from the previous format fixed. We can't cover the new format in the release notes, though you can easily see what it contains by using return { 'stdout': JSON.stringify(data)}; in handleSummary()! :smile:

This feature is only available for local k6 run tests for now, though we plan to support k6 cloud tests eventually. And, as mentioned in the snippet above, the JS helper functions that transform the summary in various formats are far from final, so keep an eye on jslib.k6.io for updates. Or, better yet, submit PRs with improvements and more transformations at https://github.com/loadimpact/jslib.k6.io :smile:

Other enhancements and UX improvements

• CI: k6 releases for Windows will now be digitally signed, which should reduce the number and severity of warnings Windows users see; the warnings would hopefully disappear altogether once Microsoft sees enough usage of the signed k6 releases to trust us (#1746). The installer and binary were also enhanced with more metadata and had their look updated with the new k6 logo and styling (#1727).
• JS: goja, the JS runtime k6 uses, was updated to its latest master version. This includes a few bugfixes and support for several new features, so --compatibility-mode=base is even more feature-rich at no additional runtime cost. We are contributing patches to goja in an effort to completely drop core.js and have the benefit of lower CPU and memory usage per VU even with --compatibility-mode=extended in the next k6 version! :tada:
• Config: integer values for duration and similar time values in the exported script options and environment variables are now treated as milliseconds. Similarly, the timeout option in http.Params can now be "stringy", e.g. "30s", "1m10s", etc. (#1738).
• HTTP: k6 now accepts ArrayBuffer values for the HTTP request body (#1776). This is a prelude/MVP for us gradually adopting ArrayBuffer for all binary data in k6 (#1020).
• Docker: We've added WORKDIR /home/k6 to our official Dockerfile (#1794).

Bugs fixed!

• HTTP: updated the golang.org/x/crypto and golang.org/x/net dependencies, which should have resolved some corner case issues with HTTP/2 connections, since k6 depends on golang.org/x/net/http2 (#1734).
• HTTP: fixed a couple of issues with blockHostnames that prevented zero-length matches for wildcards, as well as the explicit blocking of a domain and its sub-domain at the same time (#1723).
• Logs: if logs are streamed to a loki instance, k6 will now wait for them to finish being pushed before it exits - this will specifically mean that logs and errors in the init context will be propagated (#1694).
• HTTP: fixed the missing host value from http.Response.request.headers when it was explicitly set in the HTTP request params. (#1744). Thanks, @noelzubin!
• UI: fixed the lack of newline after k6 login password inputs (#1749). Thanks, @paroar!
• HTML: fixed a panic in the html.Selection.slice() method (#1756). Thanks, @asettouf!
• Summary: fixed random ordering of groups and checks in the end-of-test summary, they should now be shown in the order of their occurrence (#1788).
• Summary: the value for Rate metrics in the --summary-export JSON file was was always 0, regardless of the pass/(pass+fail) ratio (#1768).

Internals

• JS: Added automated tc39/test262 tests in our CI pipeline, so we have greater assurance that we're not breaking things when we update our JS runtime or when we finally drop core.js (#1747).
• CI: We've enabled more CI tests on Windows, now that we've switched to GitHub Actions (#1720).

Breaking changes

Some of the changes above deserve a special second mention, since they either slightly break previously documented k6 behavior, or fix previously undefined behavior and bugs that someone might have inadvertently relied on:

• Summary: --no-summary now also disables --summary-export (#1768). You can recreate the previous behavior of k6 run --no-summary --summary-export=summary.json script.js by having an empty exported handleSummary() function in your script (so that the default text summary is not shown by k6) and executing only k6 run --summary-export=summary.json script.js. Or omitting --summary-export as well and using handleSummary() as shown above.
• Config: integer values for duration and similar time values in the exported script options and environment variables are now treated as milliseconds. This was previously undefined behavior, but instead of k6 erroring out, it silently accepted and treated such values as nanoseconds (#1738).
• Docker: We've added WORKDIR /home/k6 to our official Dockerfile (#1794).

k6 v0.29.0 is here! :tada: It's a feature-packed release with tons of much-requested changes and additions, a lot of them implemented by awesome external contributors! :heart:

As promised in the previous release notes, we're trying to stick to a roughly 8-week release cycle, so you can expect the next k6 version at the start of January 2021, barring any bugfix releases before that.

New features

Initial support for gRPC (#1623)

k6 now supports unary gRPC calls via the new k6/net/grpc built-in module. Streaming RPCs are not yet supported and the JS API is in beta, so there might be slight breaking changes to the API in future k6 versions, but it's a good start on the road to fully supporting this much-requested protocol!

This is a simple example of how the new module can be used with grpcb.in:

import grpc from "k6/net/grpc";

let client = new grpc.Client();
// Download addsvc.proto for https://grpcb.in/, located at:
// https://raw.githubusercontent.com/moul/pb/master/addsvc/addsvc.proto
// and put it in the same folder as this script.
client.load(null, "addsvc.proto");

export default () => {
    client.connect("grpcb.in:9001", { timeout: "5s" });

    let response = client.invoke("addsvc.Add/Sum", {
        a: 1,
        b: 2
    });
    console.log(response.message.v); // should print 3

    client.close();
}

You can find more information and examples how to use k6's new gRPC testing capabilities in our documentation.

Huge thanks to @rogchap for adding this feature!

New options for configuring DNS resolution (#1612)

You can now control some aspects of how k6 performs DNS resolution! Previously, k6 would have cached DNS responses indefinitely (#726) and always picked the first resolved IP (#738) for all connections. This caused issues, especially when load testing services that relied on DNS for load-balancing or auto-scaling.

For technical reasons explored in (#726), k6 v0.29.0 still doesn't respect the actual TTL value of resolved IPs, that will be fixed in a future k6 version. For now, it simply allows users to specify a global static DNS TTL value and resolution strategy manually. It also has better defaults! Now, by default, the global DNS TTL value is 5 minutes and, if the DNS resolution returned multiple IPs, k6 will pick a random (preferably IPv4) one for each connection.

You can also configure this behavior with the new --dns CLI flag, the K6_DNS environment variable, or the dns script/JSON option. Three DNS resolution options are exposed in this k6 version: ttl, select, and policy.

Possible ttl values are :

• 0: no caching at all - each request will trigger a new DNS lookup.
• inf: cache any resolved IPs for the duration of the test run (the old k6 behavior).
• any time duration like 60s, 5m30s, 10m, 2h, etc.; if no unit is specified (e.g. ttl=3000), k6 assumes milliseconds. The new default value is 5m.

Possible select values are:

• first - always pick the first resolved IP (the old k6 behavior).
• random - pick a random IP for every new connection (the new default value).
• roundRobin - iterate sequentially over the resolved IPs.

Possible policy values are:

• preferIPv4: use IPv4 addresses, if available, otherwise fall back to IPv6 (the new default value).
• preferIPv6: use IPv6 addresses, if available, otherwise fall back to IPv4.
• onlyIPv4: only use IPv4 addresses, ignore any IPv6 ones.
• onlyIPv6: only use IPv6 addresses, ignore any IPv4 ones.
• any: no preference, use all addresses (the old k6 behavior).

Here are some configuration examples:

k6 run --dns "ttl=inf,select=first,policy=any" script.js # this is the old k6 behavior
K6_DNS="select=random,ttl=5m,policy=preferIPv4" k6 cloud script.js # new default behavior
# syntax for the JSON config file or for the exported script `options`:
echo '{"dns": {"select": "roundRobin", "ttl": "1h33m7s", "policy": "onlyIPv6"}}' > config.json
k6 run --config "config.json" script.js

Support for Go extensions (#1688)

After some discussions (#1353) and exploration of different approaches for Go-based k6 extensions, we've settled on adopting something very similar to caddy's extensions. In short, xk6 (modeled after xcaddy) is a small stand-alone tool that will be able to build custom k6 binaries with 3rd party extensions bundled in. The extensions can be simple Git repositories (no central infrastructure needed!) with Go modules. They will be fully compiled, not interpreted, a part of the final custom k6 binary users will be able to build with k6.

xk6 is not yet stable or documented, so any extension authors will struggle until we stabilize and document everything in the coming weeks. The important part is that the k6 changes that would allow xk6 to work were implemented in #1688, so k6 v0.29.0 is the first version compatible with xk6!

Expect more information soon, but for a brief example, xk6 will work somewhat like this:

xk6 build v0.29.0 --with github.com/k6io/xk6-k8s --with github.com/k6io/[email protected]

./k6 run some-script-with-sql-and-k8s.js

Thanks, @andremedeiros, for pushing us to add plugins in k6 and for making a valiant attempt to harness Go's poor plugin API! Thank you, @mardukbp, for pointing us towards the xcaddy approach and explaining its benefits!

Support for setting local IPs, potentially from multiple NICs (#1682)

You can now specify a list of source IPs, IP ranges and CIDRs for k6 run, from which VUs will make requests via the new --local-ips CLI flag or K6_LOCAL_IPS environment variable. The IPs will be sequentially given out to VUs, allowing you to distribute load between different local addresses. This option doesn't change anything on the OS level, so the IPs need to already be configured on the OS level in order for k6 to be able to use them.

The biggest use case for this feature is splitting the network traffic from k6 between multiple network adapters, thus potentially greatly increasing the available network throughput. For example, if you have 2 NICs, you can run k6 with --local-ips="<IP-from-first-NIC>,<IP-from-second-NIC>" to balance the traffic equally between them - half of the VUs will use the first IP and the other half will use the second. This can scale to any number of NICs, and you can repeat some local IPs to give them more traffic. For example, --local-ips="<IP1>,<IP2>,<IP3>,<IP3>" will split VUs between 3 different source IPs in a 25%:25%:50% ratio.

Thanks to @ofauchon, @srguglielmo, and @divfor for working on previous iterations of this!

New option for blocking hostnames (#1666)

You can now block network traffic by hostnames with the new --block-hostnames CLI flag / K6_BLOCK_HOSTNAMES environment variable / blockHostnames JS/JSON option. Wildcards are also supported at the beginning, allowing you to easily block a domain and all of its subdomains. For example, this will make sure k6 never attempts to connect to any k6.io subdomain (test.k6.io, test-api.k6.io, etc.) and www.example.com:

export let options = {
  blockHostnames: ["*.k6.io" , "www.example.com"],
};

Thanks to @krashanoff for implementing this feature!

UX and enhancements

• HTTP: The gjson library k6 uses for handling the HTTP Response.json(selector) behavior was updated, so we now support more modifiers like @flatten and multipaths (#1626). Thanks, @sondnm!
• HTTP: The status text returned by the server can now be accessed from the new Response.status_text field (#1649). Thanks, @lcd1232!
• HTTP: --http-debug now emits extra UUID values that can be used to match HTTP requests and their responses (#1644). Thanks, @repl-david-winiarski!
• Logging: A new allowedLabels sub-option is added to the Loki configuration (#1639).
• Cloud: when aborting a k6 cloud test with Ctrl+C, k6 will now wait for the cloud service to fully abort the test run before returning. A second Ctrl+C will cause it to immediately exit (#1647), (#1705). Thanks, @theerapatcha!
• JS: k6 will now attempt to recover from Go panics that occur in VU code, so they will be treated similarly to JS exceptions (#1697). This is just a precaution that should never be needed. panics should not happen and if one occurs, please report it in our issue tracker, since it's most likely a bug in k6.

Bugs fixed!

• JS: goja, the JS runtime k6 uses, was updated to its latest version, to fix some issues with regular expressions after its previous update (#1707).
• JS: Prevent loops with --compatibility-mode=extended when Babel can transpile the code but goja can't parse it (#1651).
• JS: Fixed a bug that rarely caused a context canceled error message to be shown (#1677).
• HTTP: Setting an empty userAgent option caused k6 to revert to the default Go User-Agent header value of Go-http-client. Now something like --user-agent='' will cause k6 to not send the User-Agent header at all (#1695).
• Cloud: k6 cloud --quiet didn't just hide the progressbar, but caused the k6 process to exit as soon as the cloud test was created, without waiting for it to finish. k6 cloud -q is now fixed to be similar to the k6 run -q behavior, but if the old behavior was wanted, something close to it can be recreated by using k6 cloud --exit-on-running (#1702).

Internals

• We switched to GitHub Actions for our CI (#1640).
• We've updated to Go 1.15 for building the official k6 binary releases (#1679)

Breaking changes

• As mentioned above, DNS resolution defaults were changed. If you want to use the old k6 behavior of always picking the first IP, no IPv4 preference, and caching DNS responses indefinitely, run k6 with --dns="ttl=inf,select=first,policy=any".

k6 v0.28.0 is here! :tada: It's a small release that adds some much requested features and a few important bugfixes!

Starting with this release, we'll be trying to stick to a new 8-week fixed release schedule for new k6 versions. This release comes ~8 weeks after v0.27.0 was released, and k6 v0.29.0 should be released in mid-November.

New features and enhancements!

Cloud execution logs (#1599)

Logs from distributed k6 cloud test runs will now be shown in the terminal that executed the k6 cloud command, as well as in the k6 cloud web app on app.k6.io! :tada: This means that, if your script contains console.log() / console.warn() / etc. calls, or some of your requests or iterations fail, you'd be able to see that and debug them much more easily! Even --http-debug data should be proxied, up to 10000 bytes per message. To prevent abuse and not to overwhelm any user terminals, cloud logs are rate-limited at 10 messages per second per instance, for now, but that should be more than enough to debug most issues!

This feature is enabled by default, though you can disable it with k6 cloud --show-logs=false script.js.

Pushing k6 logs to loki (#1576)

k6 can now push its execution logs to a loki server! This can be done via the new --log-output CLI flag or the K6_LOG_OUTPUT environment variable option. For example, k6 run --log-output "loki=https://my-loki-server/loki/api/v1/push,limit=100,level=info,msgMaxSize=10000" will push up to 100 k6 log messages per second, of severity INFO and up, truncated to 10000 bytes, to https://my-loki-server.

Optional port to host mappings (#1489)

@calavera added an extension for the host mapping feature. Now you can specify different port numbers via the hosts option, like this:

import http from 'k6/http';

export let options = {
    hosts: {
        'test.k6.io': '127.0.0.1:8080',
    },
};

Support for specifying data types to InfluxDB fields (#1395)

@TamiTakamiya added support for specifying the data type (int/float/bool/string) of fields that are emitted to InfluxDB outputs.

In order to specify the data type, you should:

• Use the environment variable K6_INFLUXDB_TAGS_AS_FIELDS, which is used to specify which k6 metric tag values should be sent as nonindexable fields (instead of tags) to an InfluxDB output. This is specified as a comma-separated string, and is now extended to optionally allow specifying a data type to each name.
• Each pair of field name and its data type is represented in the format (name):(data_type), for example, event_processing_time:int.
• One of four data types (int, float, bool and string) can be specified to one field name.
• When the colon and a data_type are omitted, for example transaction_id, it is interpreted as a string field.

A complete example can look like this: export K6_INFLUXDB_TAGS_AS_FIELDS="vu:int,iter:int,url:string,boolField:bool,floatField:float"

Note: If you have existing InfluxDB databases that contain fields whose data types are different from the ones that you want to save in future k6 test executions, you may want to create a new database or change field names as the current InfluxDB offers limited support for changing fields' data type. See the InfluxDB documentation for more details.

Support for automatic gzip-ing of the CSV output result (#1566)

@thejasbabu added support to gzip archiving the file emitted by the CSV output on the fly. To use it, simply append .gz at the end of the file name, like this: k6 run --out csv=test.csv.gz test.js

UX

• Various spacing and progress bar rendering issues were improved (#1580).
• The k6 ASCII logo was made a bit more proportional (#1615). Thanks, @rawtaz!
• The docker-compose example setup from the k6 repo now contains a built-in simple dashboard (#1610). Thanks, @jeevananthank!
• Some logs now have a source field specifying if a log comes from console, http-debug or stacktrace (when an exception has bubbled to the top of the iteration).

Bugs fixed!

• Network: IPv6 support was fixed as a part of the new hosts port mapping (#1489). Thanks, @calavera!
• Metrics: Fixed the wrong name metric tag for redirected requests (#1474).
• UI: Fixed a divide by zero panic caused by some unusual execution environments that present a TTY, but return 0 for the terminal size (#1581).
• Config: Fixed the parsing of K6_DATADOG_TAG_BLACKLIST (#1602).
• Config: Fixed marshaling of tlsCipherSuites and tlsVersion (#1603). Thanks, @berndhartzer!
• WebSockets: Fixed a ws.SetTimeout() and ws.SetInterval() panic when float values were passed (#1608).

Internals

• goja, the JavaScript runtime k6 uses, was updated to the latest version. This means that k6 with --compatibility-mode=base now supports some standard library features from ES6 (goja's PR), though no new syntax yet. In future versions we plan to drop some current core.js modules that are no longer needed, which should greatly reduce memory usage per VU (#1588).
• Go modules are now used to manage the k6 dependencies instead of dep (#1584).

Breaking changes

• k6 cloud will now proxy execution logs back to the client machine. To disable this behavior, use k6 cloud --show-logs=false.

• --http-debug request and response dumps are now emitted through the logging sub-system, to facilitate the cloud log proxying (#1577).

k6 v0.27.1 is a minor release with a few bugfixes and almost no functional changes compared to v0.27.0.

The biggest fix was resolving a panic (and some k6 login errors) when k6 was ran through git bash / Mintty on Windows (#1559).

k6 will now work in those terminals, however, if you're using git bash or Mintty as your terminal on Windows, you might not get the best user experience out of k6. Consider using a different terminal like Windows Terminal, PowerShell or Cmder. Alternatively, to work around the issues with the incompatible terminals, you can try running k6 through winpty, which should already be preinstalled in your git bash environment: winpty k6 run script.js.

If you're using the Windows Subsystem for Linux (WSL), you are probably going to get better experience by using the official Linux k6 binary or .deb package. For all other cases of running k6 on Windows, the normal k6 Windows binary / .msi package should work well.

Other minor fixes and changes:

• The Go version that k6 is compiled with was updated to 1.14.6, to incorporate the latest Go fixes (#1563).
• If the throw option is enabled, warnings for failed HTTP requests will no longer be logged to the console (#1199).
• Metric sample packets sent to the cloud with k6 run --out cloud can now be sent in parallel via the new K6_CLOUD_METRIC_PUSH_CONCURRENCY option, with a default value of 1 (#1569).
• The gracefulRampDown VU requirement calculations for the ramping-vus executor were greatly optimized for large test runs (#1567).
• Fixed a rare bug where dropped_iterations wouldn't be emitted by the per-vu-iterations executor on time due to a race (#1357).
• Metrics, including checks, from setup() and teardown(), were not correctly shown in local k6 runs (#949).

k6 v0.27.0 is here! :tada:

This is a milestone release containing a major overhaul to the execution subsystem of k6, along with many improvements and bug fixes.

New features and enhancements!

New execution engine (#1007)

After 1.5 years in the making, the k6 team is proud to release the first public version of the new execution engine, offering users new ways of modeling advanced load testing scenarios that can more closely represent real-world traffic patterns.

These new scenarios are entirely optional, and the vast majority of existing k6 scripts and options should continue to work the same as before. There are several minor breaking changes and fixes of previously undefined behavior, but please create a new issue if you find some issue we haven't explicitly noted as a breaking change.

See the documentation for details and examples, or keep reading for the summary.

New executors

Some of the currently possible script execution patterns were formalized into standalone executors:

• shared-iterations: a fixed number of iterations are "shared" by all VUs, and the test ends once all iterations are executed. This executor is equivalent to the global vus and iterations (plus optional duration) options.
• constant-vus: a fixed number of VUs execute as many iterations as possible for a specified amount of time. This executor is equivalent to the global vus and duration options.
• ramping-vus: a variable number of VUs execute as many iterations as possible for a specified amount of time. This executor is equivalent to the global stages option.
• externally-controlled: control and scale execution at runtime via k6's REST API or the CLI.

You'd still be able to use the global vus, iterations, duration, and stages options, they are not deprecated! They are just transparently converted to one of the above executors underneath the hood. And if your test run needs just a single, simple scenario, you may never need to use more than these shortcut options. For more complicated use cases however, you can now fine-tune any of these executors with additional options, and use multiple different executors in the same test run, via the new scenarios option, described below.

Additionally, besides the 4 "old" executor types, there are 3 new executors, added to support some of the most frequently requested load testing scenarios that were previously difficult or impossible to model in k6:

• per-vu-iterations: each VU executes a fixed number of iterations (#381).
• constant-arrival-rate: iterations are started at a specified fixed rate, for a specified duration. This allows k6 to dynamically change the amount of active VUs during a test run, to achieve the specified amount of iterations per period. This can be very useful for a more accurate representation of RPS (requests per second), for example. See #550 for details.
• ramping-arrival-rate: a variable number of iterations are executed in a specified period of time. This is similar to the ramping VUs executor, but instead of specifying how many VUs should loop through the script at any given point in time, the iterations per second k6 should execute at that point in time can be specified.

It's important to also note that all of these executors, except the externally-controlled one, can be used both in local k6 execution with k6 run, and in the distributed cloud execution with k6 cloud. This even includes "old" executors that were previously unavailable in the cloud, like the shared-iterations one. Now, you can execute something like k6 cloud --iterations 10000 --vus 100 script.js without any issues.

Execution scenarios and executor options

Multiple execution scenarios can now be configured in a single test run via the new scenarios option. These scenarios can run both sequentially and in parallel, and can independently execute different script functions, have different executor types and execution options, and have custom environment variables and metrics tags.

An example using 3 scenarios:

import http from 'k6/http';
import { sleep } from 'k6';

export let options = {
    scenarios: {
        my_web_test: { // some arbitrary scenario name
            executor: 'constant-vus',
            vus: 50,
            duration: '5m',
            gracefulStop: '0s', // do not wait for iterations to finish in the end
            tags: { test_type: 'website' }, // extra tags for the metrics generated by this scenario
            exec: 'webtest', // the function this scenario will execute
        },
        my_api_test_1: {
            executor: 'constant-arrival-rate',
            rate: 90, timeUnit: '1m', // 90 iterations per minute, i.e. 1.5 RPS
            duration: '5m',
            preAllocatedVUs: 10, // the size of the VU (i.e. worker) pool for this scenario
            maxVUs: 10, // we don't want to allocate more VUs mid-test in this scenario

            tags: { test_type: 'api' }, // different extra metric tags for this scenario
            env: { MY_CROC_ID: '1' }, // and we can specify extra environment variables as well!
            exec: 'apitest', // this scenario is executing different code than the one above!
        },
        my_api_test_2: {
            executor: 'ramping-arrival-rate',
            startTime: '30s', // the ramping API test starts a little later
            startRate: 50, timeUnit: '1s', // we start at 50 iterations per second
            stages: [
                { target: 200, duration: '30s' }, // go from 50 to 200 iters/s in the first 30 seconds
                { target: 200, duration: '3m30s' }, // hold at 200 iters/s for 3.5 minutes
                { target: 0, duration: '30s' }, // ramp down back to 0 iters/s over the last 30 second
            ],
            preAllocatedVUs: 50, // how large the initial pool of VUs would be
            maxVUs: 100, // if the preAllocatedVUs are not enough, we can initialize more

            tags: { test_type: 'api' }, // different extra metric tags for this scenario
            env: { MY_CROC_ID: '2' }, // same function, different environment variables
            exec: 'apitest', // same function as the scenario above, but with different env vars
        },
    },
    discardResponseBodies: true,
    thresholds: {
        // we can set different thresholds for the different scenarios because
        // of the extra metric tags we set!
        'http_req_duration{test_type:api}': ['p(95)<250', 'p(99)<350'],
        'http_req_duration{test_type:website}': ['p(99)<500'],
        // we can reference the scenario names as well
        'http_req_duration{scenario:my_api_test_2}': ['p(99)<300'],
    }
};

export function webtest() {
    http.get('https://test.k6.io/contacts.php');
    sleep(Math.random() * 2);
}

export function apitest() {
    http.get(`https://test-api.k6.io/public/crocodiles/${__ENV.MY_CROC_ID}/`);
    // no need for sleep() here, the iteration pacing will be controlled by the
    // arrival-rate executors above!
}

As shown in the example above and the documentation, all executors have some additional options that improve their flexibility and facilitate code reuse, especially in multi-scenario test runs:

• Each executor has a startTime property, which defines at what time, relative to the beginning of the whole test run, the scenario will start being executed.
• Executors have a new gracefulStop property that allows for iterations to complete gracefully for some amount of time after the normal executor duration is over (#879, #1033). The ramping-vus executor additionally also has gracefulRampDown, to give iterations time to finish when VUs are ramped down. The default value for both options is 30s, so it's a slight breaking change, but the old behavior of immediately interrupting iterations can easily be restored by setting these options to 0s.
• Different executors can execute different functions other than the default exported one. This can be specified by the exec option in each scenarios config, and allows for more flexibility in organizing your tests, easier code reuse, building test suites, etc.
• To allow for even greater script flexibility and code reuse, you can specify different environment variables and tags in each scenario, via the new env and tags executor options respectively.
• k6 may now emit a new dropped_iterations metric in the shared-iterations, per-vu-iterations, constant-arrival-rate and ramping-arrival-rate executors; this is done if it can't run an iteration on time, depending on the configured rates (for the arrival-rate executors) or scenario maxDuration (for the iteration-based executors), so it's generally a sign of a poor config or an overloaded system under test (#1529).

We've also introduced new --execution-segment and --execution-segment-sequence options, which allow for relatively easy partitioning of test runs across multiple k6 instances. Initially this applies to the test execution (all new executor types are supported!), but opens the door to test data partitioning, an often requested feature. See #997 for more details.

UX

• CLI: There are separate descriptions and real-time thread-safe progress bars for each individual executor.
• CLI: Improve module import error message (#1439).
• JS: The __VU variable is now available in the script init context, allowing easier splitting of test input data per VU and reducing RAM usage (#889).
• A new method to stop engine execution via the REST API (#1352). Thanks @hynd!

Bugs fixed!

• CLI: Stop --http-debug from exiting k6 on request dump error (#1402). Thanks @berndhartzer!
• CLI: JSON output is now less noisy (#1469). Thanks @michiboo!
• CLI: k6 doesn't exit when using iterations with stages (#812).
• CLI: Mismatch in check counts in the end-of-test summary (#1033).
• Config: Better validation of stages (#875).
• JS: Rare panics in goja (#867,#1552).
• HTTP: Fix request timeout and wrong context used for pushing metrics (#1260).
• Execution: Fix sometimes skipping iterations with a context cancelled error when rapidly ramping up and down (#1283).
• WebSockets: Fix possible connection hanging when the main context is cancelled (#1260).
• WebSockets: Avoid leaking goroutines (#1513).
• WebSockets: Emit WS metrics as soon as possible instead of when the connection is closed (#885).

Internals

• As mentioned above, #1007 was almost a complete re-write of the execution scheduling parts of k6. This involved deep architectural changes in how test runs are executed and improved overall code coverage by around 2%.
• Switched to Go 1.14 for building and testing k6, bringing some fixes and performance improvements.
• WebSockets: Updated gorilla/websocket library bringing minor performance improvements (#1515).
• Code cleanup and formatting. Thanks @thinkerou!

Breaking changes

• Execution config options (scenarios, stages, iterations, duration) from "upper" config layers overwrite execution options from "lower" (i.e. CLI flags > environment variables > JS options > JSON options) config layers. For example, the --iterations CLI flag will overwrite any execution options set as environment variables (e.g. K6_DURATION, K6_STAGES, etc.) or script options (stages: [ /* ... */], scenarios: { /* ... */ }, etc. ).

• Previously, if the iterations and vus script options were specified, but duration was not, the script could have ran practically indefinitely, given a sufficiently large number or length of the used iterations. There was no implicit or explicit time limit, one of the reasons this execution pattern was not allowed in k6 cloud test runs before. From k6 v0.27.0, by default, if the specified iterations haven't finished, these scripts will abort after 10 minutes, the default maxDuration value of shared-iterations executors. This default value can easily be changed by either setting the maxDuration option in the corresponding scenarios entry, or, if just the execution shortcuts were used to run the script, by setting the duration / --duration / K6_DURATION script option.

• Previously, all iterations were interruptible - as soon as the specified duration expired, or when VUs were ramped down in a stage, any running iterations were interrupted. Now all executors besides the externally-controlled one have a gracefulStop period of 30s by default (#898). Additionally, the ramping-vus executor has a gracefulRampDown parameter that configures the ramp-down grace period. For those periods, no new iterations will be started by the executors, but any currently running iterations will be allowed up to the specified periods to finish their execution.

• Using different execution config options on the same level is now a configuration conflict error and will abort the script. For example, executing k6 run --duration 10s --stages 5s:20 script.js won't work (#812). The only exception is combining duration and iterations, which will result in a shared-iterations executor with the specified non-default maxDuration (#1058).

• The k6 REST API for controlling script execution (i.e. the k6 pause, k6 scale commands) now only works when a externally-controlled executor is configured in the scenarios config. The initial pausing of a test (i.e. k6 run --paused script.js) still works with all executor types, but once the test is started with k6 resume (or the corresponding REST API call), it can't be paused again unless only the externally-controlled executor is used.

• Previously, running a script with k6 run --paused script.js would have still executed the script's setup() function (if it was present and wasn't explicitly disabled with --no-setup) and paused immediately after. Now, k6 will pause before it executes setup().

• Previously, if you ramped down and ramped up VUs via stages, the __VU global variables would have been incremented on the ramp-ups. This will no longer happen, the max value of __VU across a test run will never exceed the number of initialized VUs.

• The vusMax / K6_VUS_MAX / -m / --max option is deprecated - it was previously used for the control of the initialized VUs by the REST API. Since that has now been restricted to the externally-controlled executor, the equivalent option there is called maxVUs.

• Tests with infinite duration are now only possible via the externally-controlled executor.

• k6 will now exit with an error if --address is specified but the API server is unable to start. Previously this would've resulted in a warning message.

• The format returned by Date.prototype.toUTCString() was changed from Thu Jan 01 1970 00:00:00 GMT+0000 (UTC) to Thu, 01 Jan 1970 00:00:00 GMT. This is a fix that aligns it with the ECMAScript spec. See https://github.com/dop251/goja/issues/119 .

• The default setupTimeout and teardownTimeout values were changed from 10s to 60s (#1356).

k6 v0.26.2 is a minor release that updates the used Go version for the Windows builds to Go 1.13.8. Due to an oversight, previous v0.26 k6 builds for Windows used an old Go version, while builds of other OSes used the correct one. This is meant to address an issue in the Go net/http package: https://github.com/golang/go/issues/34285 .

There are no functional changes compared to v0.26.1.

k6 v0.26.1 is here! This is a minor release that supports the rebranding of LoadImpact to k6, the new k6.io website, and the new k6 cloud service! :tada:

In practical terms, all that it means for k6 is that the URLs for cloud tests will point to https://app.k6.io, instead of https://app.loadimpact.com. The old URLs (and old k6 versions) will still continue to work - for the next 3 months the old app and the new one would work in parallel, and after that period the old app will redirect to the new one. Nothing changes in regards to the k6 open source project and our commitment to it!

You can find more information about the rebranding in our blog post about it: https://k6.io/blog/load-impact-rebranding-to-k6

Changes in this release compared to v0.26.0:

• Fix how HTTP request timeouts are specified internally. This is not a bug in current k6 releases, it only affects k6 if it is compiled with Go 1.14, which at this time is still not officially released. (#1261)
• Improve the official docker image to use an unprivileged user. Thanks, @funkypenguin! (#1314)
• Fix the unintentional sharing of __ENV between VUs, which could result in data races and crashes of k6. (#1329)
• Update cloud URLs to point to https://app.k6.io instead of https://app.loadimpact.com. (#1335)

k6 v0.26.0 is here! :tada:

This release contains mostly bug fixes, though it also has several new features and enhancements! They include a new JS compatibility mode option, exporting the end-of-test summary to a JSON report file, speedups to the InfluxDB and JSON outputs, http.batch() improvements, a brand new CSV output, multiple layered HTTP response body decompression, being able to use console in the init context, a new optional column in the summary, and Docker improvements!

Thanks to @Sirozha1337, @openmohan, @MMartyn, @KajdeMunter, @dmitrytokarev and @dimatock for contributing to this release!

New features and enhancements!

A new JavaScript compatibility mode option (#1206)

This adds a way to disable the automatic script transformation by Babel (v6.4.2) and loading of core-js (v2) polyfills, bundled in k6. With the new base compatibility mode, k6 will instead rely only on the goja runtime and what is built into k6. This can be configured through the new --compatibility-mode CLI flag and the K6_COMPATIBILITY_MODE environment variable. The possible values currently are:

• extended: this is the default and current compatibility mode, which uses Babel and core.js to achieve ES6+ compatibility.
• base: an optional mode that disables loading of Babel and core.js, running scripts with only goja's native ES5.1+ compatibility. If the test scripts don't require ES6 compatibility (e.g. they were previously transformed by Babel), this option can be used to reduce RAM usage during test runs.

More info what this means can be found in the documentation.

Our benchmarks show a considerable drop in memory usage - around 80% for simple scripts, and around 50% in the case of 2MB script with a lot of static data in it. The CPU usage is mostly unchanged, except that k6 initializes test runs a lot faster. All of those benefits will be most noticeable if k6 is used with big number of VUs (1k+). More performance comparisons can be found in #1167.

JSON export of the end-of-test summary report (#1168)

This returns (from the very early days of k6) the ability to output the data from the end of test summary in a machine-readable JSON file. This report can be enabled by the --summary-export <file_path> CLI flag or the K6_SUMMARY_EXPORT environment variable. The resulting JSON file will include data for all test metrics, checks and thresholds.

New CSV output (#1067)

There is an entirely new csv output that can be enabled by using the --out csv CLI flag. There are two things that can be configured: the output file with K6_CSV_FILENAME (by default it's file.csv), and the interval of pushing metrics to disk, which is configured with K6_CSV_SAVE_INTERVAL (1 second by default). Both of those can be configured by the CLI as well: --out csv=somefile.csv will output to somefile.csv and --out file_name=somefile.csv,save_interval=2s will output again to somefile.csv, but will flush the data every 2 seconds instead of every second.

The first line of the output is the names of columns and looks like:

metric_name,timestamp,metric_value,check,error,error_code,group,method,name,proto,status,subproto,tls_version,url,extra_tags
http_reqs,1573131887,1.000000,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/,
http_req_duration,1573131887,116.774321,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/,
http_req_blocked,1573131887,148.691247,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/,
http_req_connecting,1573131887,112.593448,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/,

All thanks to @Sirozha1337!

JSON output optimizations (#1114)

The JSON output no longer blocks the goroutine sending samples to the file, but instead (like all other outputs) buffers the samples and writes them at regular intervals (100ms and is currently not configurable). It also uses a slightly faster way of encoding the data, which should decrease the memory usage by a small amount.

Another improvement is the ability to compress the generated JSON file by simply adding .gz to the end of the file name. Compressed files are typically 30x smaller.

InfluxDB output improvements (#1113)

The InfluxDB output has been updated to use less memory and try to send smaller and consistent chunks of data to InfluxDB, in order to not drop packets and be more efficient. This is primarily done by sending data in parallel, as this seems to be better from a performance perspective, and more importantly, queuing data in separate packets, so that we don't send the data for a big time period all at once. Also, the used library was updated, which also decreased the memory usage.

Two new options were added:

• K6_INFLUXDB_PUSH_INTERVAL - configures at what interval the collected data is queued to be sent to InfluxDB. By default this is "1s".
• K6_INFLUXDB_CONCURRENT_WRITES - configures the number of concurrent write calls to InfluxDB. If this limit is reached the next writes will be queued and made when a slot is freed. By default this is 10.

console is now available in the init context (#982):

This wasn't supported for the longest time, which made debugging things outside of VU code much harder, but now it's here! :tada:

In order to get this feature shipped in a timely manner, it currently has a known bug. The output of console calls in the init context will always be written to the stderr, even if the --console-output option is specified. This bug is tracked in https://github.com/loadimpact/k6/issues/1131

HTTP response body decompression with multiple layered algorithms (#1125)

In v0.25.0 compressing bodies was added and it had support for multiple layered algorithms. Now this is also true for decompressing bodies when k6 gets them as responses.

New optional count column in the end-of-test summary (#1143)

The --summary-trend-stats now also recognizes count as a valid column and will output the count of samples in all Trend metrics. This could be especially useful for custom Trend metrics, since with them you no longer need to specify a separate accompanying Counter metric.

Docker Compose refactor (#1183)

The example docker-compose that enabled easy running of InfluxDB+Grafana+k6 was refactored and all the images were updated to use the latest stable versions.

Thanks, @KajdeMunter!

Also the k6 Dockerfile Alpine version was bumped to 3.10. Thanks @dmitrytokarev!

http.batch() improvements and optimizations (#1259)

We made several small improvements to the mechanism for executing multiple HTTP requests simultaneously from a single VU:

• Calling http.batch() should now be more efficient, especially for many requests, because of reduced locking, type conversions, and goroutine spawning.
• The default value for batchPerHost has been reduced from 0 (unlimited) to 6, to more closely match browser behavior. The default value for the batch option remains unchanged at 20.
• Calling http.batch(arg), where arg is an array, would now return an array. Previously, this would have returned an object with integer keys, as explained in #767... Now http.batch() will return an array when you pass it an array, and return an object when you pass an object.

UX

• Better timeout messages for setup and teardown timeouts, including hints on how to fix them. (#1173)
• When a folder is passed to open(), the resulting error message will now include the path to the specified folder. (#1238)
• The k6 version output will now include more information - the git commit it was built from (in most cases), as well as the used Go version and architecture. (#1235)

Bugs fixed!

• Cloud: Stop sending metrics to the cloud output when the cloud returns that you have reached the limit. (#1130)
• JS: Fail a check if an uncaught error is thrown inside of it. (#1137)
• HTTP: Replace any user credentials in the metric sample tags with * when emitting HTTP metrics. (#1132)
• WS: Many fixes:
  • return an error instead of panicking if an error occurs during the making of the WebSocket connection (#1127)
  • calling the error handler on an error when closing the WebSocket, instead of calling with a null (#1118)
  • correctly handle server initiated close (#1186)
• JSON: Better error messages when parsing JSON fails. Now telling you at which line and row the error is instead of just the offset. Thanks, @openmohan! (#905)
• HTTP: Use Request's GetBody in order to be able to get the body multiple times for a single request as needed in 308 redirects of posts and if the server sends GOAWAY with no error. (#1093)
• JS: Don't export internal go struct fields of script options.(#1151)
• JS: Ignore minIterationDuration for setup and teardown. (#1175)
• HTTP: Return error on any request that returns 101 status code as k6 currently doesn't support any protocol upgrade behavior. (#1172)
• HTTP: Correctly capture TCP reset by peer and broken pipe errors and give them the appropriate error_code metric tag values. (#1164)
• Config: Don't interpret non-K6_ prefixed environment variables as k6 configuration, most notably DURATION and ITERATIONS. (#1215)
• JS/html: Selection.map was not wrapping the nodes it was outputting, which lead to wrongly using the internal Goquery.Selection instead of k6's Selection. Thanks to @MMartyn for reporting this! (#1198)
• HTTP: When there are redirects, k6 will now correctly set the cookie for the current URL, instead of for the one the current response is redirecting to. Thanks @dimatock! (#1201)
• Cloud: Add token to make calls to the cloud API idempotent. (#1208)
• Cloud: Improve aggregation of HTTP metrics for requests to different URLs, but with the same explicitly set name tag. (#1220)
• Cloud: Fix a bug where you weren't able to run a script, outputting to cloud, if it was using the shortcut URLs for github/cdnjs. (#1237)
• Config: The previous default value for batchPerHost of 20 wasn't propagated properly and was instead 0. (#1264)

Internals

• CI: Stop using external service for testing WebSockets (#1138) and remove the last use of the external httpbin.org. (#1213)
• Switched to Go 1.13.5 for building and testing k6, removed official support for 1.11.
• CI: Fix a test on MacOS. (#1142)
• CI: Fixing flaky tests. (#1149, #1223)
• Drop an external dependency for getting user's configdir. (#1162)
• Use bitmask for checking whether system tags are enabled, adding some small speedup where this is required. (#1148)
• Update envconfig as it was very old and the newer versions had fixes and features we want. (#1214)
• Metrics: Emit iterations as part of netext.NetTrail, instead of as a standalone one. Also cutting down on amount of bytes we sent to the cloud output. (#1203)
• JS: goja has been updated to the latest master version (commit 007eef3) (#1259)
• All official binary packages now are built with -trimpath and CGO_ENABLED=0. Previously the GitHub release assets were built with CGO_ENABLED=0, making them unsuitable for non-glibc systems (like Alpine Linux). (#1244, #1245)

Breaking changes

• The output of k6 version is different. It now contains not only the k6 version number, but also information about the git commit, build date, Go version and system architecture. For example, if previously the output of k6 version looked like k6 v0.25.1, now it is like this: k6 v0.26.0 (2019-12-16T10:58:59+0000/v0.26.0-0-gaeec9a7f, go1.13.5, linux/amd64). (#1235)

• We had to make a few minor breaking changes in the course of improving http.batch() (#1259):

  • The default value for batchPerHost has been reduced from 0 (unlimited) to 6, to more closely match browser behavior. The default value for the batch option remains unchanged at 20.
  • Calling http.batch(arg), where arg is an array, would now return an array. Previously, this would have returned an object with integer keys, as explained in #767... Now http.batch() will return an array when you pass it an array, and return an object when you pass an object.

A minor release that fixes some of the issues in the v0.25.0 release.

Bugs fixed!

• Config: Properly handle the systemTags JS/JSON option and the K6_SYSTEM_TAGS environment variable. Thanks, @cuonglm! (#1092)
• HTTP: Fix how request bodies are internally specified so we can properly handle redirects and can retry some HTTP/2 requests. (#1093)
• HTTP: Fix the handling of response decoding errors and slightly improve the digest auth and --http-debug code. (#1102)
• HTTP: Always set the correct Content-Length header for all requests. (#1106)
• JS: Fix a panic when executing archive bundles for scripts with unsuccessfull import / require() calls. (#1097)
• JS: Fix some issues related to the handling of exports corner cases. (#1099)

k6 v0.25.0 is here! :tada:

This release contains mostly bug fixes, though it also has a few new features, enhancements, and performance improvements. These include HTTP request compression, brotli and zstd support, massive VU RAM usage and initialization time decreases, support for importing files via https and file URLs, and opt-in TLS 1.3 support.

Thanks to @THoelzel, @matlockx, @bookmoons, @cuonglm, and @imiric for contributing to this release!

New features and enhancements!

HTTP: request body compression + brotli and zstd decompression (#989, #1082)

Now k6 can compress the body of any HTTP request before sending it (#989). That can be enabled by setting the new compression option in the http.Params object. Doing so will cause k6 to transparently compress the supplied request body and correctly set both Content-Encoding and Content-Length, unless they were manually set in the request headers by the user. The currently supported algorithms are deflate, gzip, brotli and zstd, as well as any combination of them separated by commas (,).

k6 now also transparently decompresses brotli and zstd HTTP responses - previously only deflate and gzip were supported. Thanks, @imiric! (#1082)

import http from 'k6/http';
import { check } from "k6";

export default function () {
    // Test gzip compression
    let gzippedReqResp = http.post("https://httpbin.org/post", "foobar".repeat(1000), { compression: "gzip" });
    check(gzippedReqResp, {
        "request gzip content-encoding": (r) => r.json().headers["Content-Encoding"] === "gzip",
        "actually compressed body": (r) => r.json().data.length < 200,
    });

    // Test br decompression
    let brotliResp = http.get("https://httpbin.org/brotli", {
        headers: {
            "Accept-Encoding": "gzip, deflate, br"
        },
    });
    check(brotliResp, {
        "br content-encoding header": (r) => r.headers["Content-Encoding"] === "br",
        "br confirmed in body": (r) => r.json().brotli === true,
    });

    // Test zstd decompression
    let zstdResp = http.get("https://facebook.com/", {
        headers: {
            "Accept-Encoding": "zstd"
        },
    });
    check(zstdResp, {
        "zstd content-encoding header": (r) => r.headers["Content-Encoding"] === "zstd",
        "readable HTML in body": (r) => r.body.includes("html"),
    });
};

Performance improvement: reuse the parsed core-js library across VUs (#1038)

k6 uses the awesome core-js library to support new JavaScript features. It is included as a polyfill in each VU (i.e. JS runtime) and previously, it was parsed anew for every VU initialization. Now, the parsing result is cached after the first time and shared between VUs, leading to over 2x reduction of VU memory usage and initialization times for simple scripts!

Thanks, @matlockx, for noticing this opportunity for massive optimization!

JS files can now be imported via https and file URLs (#1059)

Previously, k6 had a mechanism for importing files via HTTPS URLs, but required that the used URLs not contain the https scheme. As a move to align k6 more closely with the rest of the JS ecosystem, we now allow and encourage users to use full URLs with a scheme (e.g. import fromurlencoded from "https://jslib.k6.io/form-urlencoded/3.0.0/index.js") when they want to load remote files. file URLs are also supported as another way to load local modules (normal absolute and relative file paths still work) from the local system, which may be especially useful for Windows scripts.

The old way of importing remote scripts from scheme-less URLs is still supported, though except for the GitHub and cdnjs shortcut loaders, it is in the process of deprecation and will result in a warning.

Opt-in support for TLS 1.3 and more TLS ciphers (#1084)

Following its opt-in support in Go 1.12, you can now choose to enable support for TLS 1.3 in your k6 scripts. It won't be used by default, but you can enable it by setting the tlsVersion (or it's max sub-option) to tls1.3:

import http from 'k6/http';
import { check } from "k6";

export let options = {
    tlsVersion: {
        min: "tls1.2",
        max: "tls1.3",
    }
};

export default function () {
    let resp = http.get("https://www.howsmyssl.com/a/check");
    check(resp, {
        "status is 200": (resp) => resp.status === 200,
        "tls 1.3": (resp) => resp.json().tls_version === "TLS 1.3",
    });
};

Also, all cipher suites supported by Go 1.12 are now supported by k6 as well. Thanks, @cuonglm!

Bugs fixed!

• JS: Many fixes for open(): (#965)

  • don't panic with an empty filename ("")
  • don't make HTTP requests (#963)
  • correctly open simple filenames like "file.json" and paths such as "relative/path/to.txt" as relative (to the current working directory) paths; previously they had to start with a dot (i.e. "./relative/path/to.txt") for that to happen
  • windows: work with paths starting with / or \ as absolute from the current drive

• HTTP: Correctly always set response.url to be the URL that was ultimately fetched (i.e. after any potential redirects), even if there were non http errors. (#990)

• HTTP: Correctly detect connection refused errors on dial. (#998)

• JS: Run imports once per VU. (#975, #1040)

• Config: Fix blacklistIPs JS configuration. Thanks, @THoelzel! (#1004)

• HTTP: Fix a bunch of HTTP measurement and handling issues (#1047)

  • the http_req_receiving metric was measured incorrectly (#1041)
  • binary response bodies could get mangled in an http.batch() call (#1044)
  • timed out requests could produce wrong metrics (#925)

• JS: Many fixes for importing files and for URL imports in archives. (#1059)

• Config: Stop saving and ignore the derived execution values, which were wrongly saved in archive bundles' metadata.json by k6 v0.24.0. (#1057, #1076)

• Config: Fix handling of commas in environment variable values specified as CLI flags. (#1077)

Internals

• CI: removed the gometalinter check in CircleCI, since that project was deprecated and now exclusively rely on golangci-lint. (#1039)
• Archive bundles: The support for URL imports included a lot of refactoring and internal k6 changes. This included significant changes in the structure of .tar archive bundles. k6 v0.25.0 is backwards compatible and can execute bundles generated by older k6 versions, but the reverse is not true. (#1059)
• Archive bundles: The k6 version and the operating system are now saved in the archive bundles' metadata.json file. (#1057, #1059)

Breaking changes

• Previously, the Content-Length header value was always automatically set by k6 - if the header value was manually specified by the user, it would have been ignored and silently overwritten. Now, k6 would set the Content-Length value only if it wasn't already set by the user. (#989, #1094)

v0.24.0 is here! :tada:

Another intermediary release that was mostly focused on refactoring and bugfixes, but also has quite a few new features, including the ability to output metrics to StatsD and Datadog!

Thanks to @cheesedosa, @ivoreis, @bookmoons, and @oboukili for contributing to this release!

New Features!

Redirect console messages to a file (#833)

You can now specify a file to which all things logged by console.log() and other console methods will get written to. The CLI flag to specify the output file path is --console-output, and you can also do it via the K6_CONSOLE_OUTPUT environment variable. For security reasons, there's no way to configure this from inside of the script.

Thanks to @cheesedosa for both proposing and implementing this!

New result outputs: StatsD and Datadog (#915)

You can now output any metrics k6 collects to StatsD or Datadog by running k6 run --out statsd script.js or k6 run --out datadog script.js respectively. Both are very similar, but Datadog has a concept of metric tags, the key-value metadata pairs that will allow you to distinguish between requests for different URLs, response statuses, different groups, etc.

Some details:

• By default both outputs send metrics to a local agent listening on localhost:8125 (currently only UDP is supported as a transport). You can change this address via the K6_DATADOG_ADDR or K6_STATSD_ADDR environment variables, by setting their values in the format of address:port.
• The new outputs also support adding a namespace - a prefix before all the metric names. You can set it via the K6_DATADOG_NAMESPACE or K6_STATSD_NAMESPACE environment variables respectively. Its default value is k6. - notice the dot at the end.
• You can configure how often data batches are sent via the K6_STATSD_PUSH_INTERVAL / K6_DATADOG_PUSH_INTEVAL environment variables. The default value is 1s.
• Another performance tweak can be done by changing the default buffer size of 20 through K6_STATSD_BUFFER_SIZE / K6_DATADOG_BUFFER_SIZE.
• In the case of Datadog, there is an additional configuration K6_DATADOG_TAG_BLACKLIST, which by default is equal to `` (nothing). This is a comma separated list of tags that should NOT be sent to Datadog. All other metric tags that k6 emits will be sent.

Thanks to @ivoreis for their work on this!

k6/crypto: random bytes method (#922)

This feature adds a method to return an array with a number of cryptographically random bytes. It will either return exactly the amount of bytes requested or will throw an exception if something went wrong.

import crypto from "k6/crypto";

export default function() {
    var bytes = crypto.randomBytes(42);
}

Thanks to @bookmoons for their work on this!

k6/crypto: add a binary output encoding to the crypto functions (#952)

Besides hex and base64, you can now also use binary as the encoding parameter for the k6 crypto hashing and HMAC functions.

New feature: unified error codes (#907)

Error codes are unique numbers that can be used to identify and handle different application and network errors more easily. For the moment, these error codes are applicable only for errors that happen during HTTP requests, but they will be reused and extended to support other protocols in future k6 releases.

When an error occurs, its code is determined and returned as both the error_code field of the http.Response object, and also attached as the error_code tag to any metrics associated with that request. Additionally, for more details, the error metric tag and http.Response field will still contain the actual string error message.

Error codes for different errors are as distinct as possible, but for easier handling and grouping, codes in different error categories are also grouped in broad ranges. The current error code ranges are:

• 1000-1099 - General errors
• 1100-1199 - DNS errors
• 1200-1299 - TCP errors
• 1300-1399 - TLS errors
• 1400-1499 - HTTP 4xx errors
• 1500-1599 - HTTP 5xx errors
• 1600-1699 - HTTP/2 specific errors

For a list of all current error codes, see the docs page here.

Internals

• Improvements in the integration with loadimpact.com. (#910 and #934)
• Most of the HTTP request code has been refactored out of the js packages and is now independent from the goja JS runtime. This was done mostly so we can implement the error codes feature (#907), but will allow us more flexibility in the future. (#928)
• As a preparation for the upcoming big refactoring of how VUs are scheduled in k6, including the arrival-rate based execution, we've added the future execution configuration framework. It currently doesn't do anything besides warn users that use execution option combinations that won't be supported in future k6 versions. See the Breaking Changes section in these release notes for more information. (#913)
• Switched to golangci-lint via golangci.com for code linting in this repo. The gometalinter check in CircleCI is still enabled as well, but it will be removed in the following few weeks. (#943)
• Switched to Go 1.12.1 for building and testing k6, removed official support for 1.10. (#944 and #966)

Bugs fixed!

• JS: Consistently report setup/teardown timeouts as such and switch the error message to be more expressive. (#890)
• JS: Correctly exit with non zero exit code when setup or teardown timeouts. (#892)
• Thresholds: When outputting metrics to loadimpact.com, fix the incorrect reporting of threshold statuses at the end of the test. (#894)

• UX: --quiet/-q doesn't hide the summary stats at the end of the test. When necessary, they can still be hidden via the explicit --no-summary flag. Thanks, @oboukili! (#937)

Breaking changes

None in this release, but in preparation for the next one, some execution option combinations will emit warnings, since they will no longer be supported in future k6 releases. Specifically, you won't be able to simultaneously run k6 with stages and duration set, or with iterations and stages, or with duration and iterations, or with all three. These VU schedulers (and much more, including arrival-rate based ones!) will still be supported in future k6 releases. They will just be independent from each other, unlike their current implementation where there's one scheduler with 3 different conflicting constraints.

A minor release that fixes some of the issues in the v0.23.0 release.

Bugs fixed!

• Cloud: Fixed the interaction between the environment variable and JSON-based configuration, and the Load Impact specific env.loadimpact JS options. Now only the projectID, name and token fields will be populated (without overriding other fields) when executing scripts with k6 cloud, and taken into account when sending metrics to Load Impact Insights with k6 run -o cloud. (#848, #871, #872)
• JS: Fixed a Babel transformation issue that caused closing brackets to sometimes be commented out. (#853)
• JS: Fixed environment variable propagation when executing script bundles. (#853)
• HAR converter: Fixed a panic due to a missing nil check. (#861)
• Cloud: Limit the amount of samples that k6 sends in a single package to the ingest by splitting them up. (#860)
• Metrics: Fix the incorrect tracing of some corner case HTTP requests that resulted in negative or hugely positive metric values. (#862)

v0.23.0 is here! :tada:

Hopefully this is the last intermediary release before v1.0.0. It is a bit light on new features, but it includes a lot of bug fixes and minor improvements! Also, the latest Docker tag will point to this release until we release the next stable one. Users wanting to use the bleeding edge k6 features can do that via the new master docker tag, which is pushed by every CI build of the git master branch.

Thanks to @sherrman, @ofauchon, @AndriiChuzhynov, @entone, @mariolopjr, and @tkbky for contributing to this release!

To see what's left for the v1.0.0 release, check out this milestone!

Also, have a look at our roadmap for what's up ahead, beyond the v1.0 release.

New Features!

New option: No Cookies Reset (#729)

A new option has been added that disables the default behavior of resetting the cookie jar after each VU iteration. If it's enabled, saved cookies will be persisted across VU iterations. For the moment there's no CLI flag for this option, instead it can only be set via the noCookiesReset key from the exported script options or via the K6_NO_COOKIES_RESET environment variable.

k6/http: New options to discard the response body or to specify its type (#742 and #749)

You can now specify what the type of an HTTP response's body should be with the new responseType request option. The possible values for it are text (the default), binary and none. The default text response type is backward-compatible, it doesn't change the current k6 behavior of returning the body attribute of the http/Response object as a string. It's well suited for working with web pages, text-based APIs and similar HTTP responses, but it can be unsuitable when dealing with binary files.

That's mostly because JavaScript strings are encoded with UTF-16 and converting binary data to it will frequently mangle some of the data. The new binary response type allows us to avoid that, it causes k6 to return the HTTP response's body as a byte array. This allows us to deal with the binary data without mangling it:

import http from 'k6/http';
import { sha256 } from 'k6/crypto';

export default function () {
    const expectedLogoHash = "fce7a09dde7c25b9822eca8438b7a5c397c2709e280e8e50f04d98bc8a66f4d9";

    let resp = http.get("http://test.loadimpact.com/images/logo.png", { responseType: "binary" });
    let logoHash = sha256(resp.body, "hex");
    if (logoHash !== expectedLogoHash) {
        throw new Error(`Expected logo hash to be ${expectedLogoHash} but it was ${logoHash}`);
    }
    http.post("https://httpbin.org/post", resp.body);
};

Saving HTTP response bodies is generally useful, especially when we need to use them (or parts of them) in subsequent requests. But in many cases it makes little to no sense to spend memory on saving the response body. For example, when requesting static website assets (JS, CSS, images etc.) or web pages without needed information, the actual file contents rarely matter when running load tests.

For cases like that, the value none for the responseType option allows k6 to discard incoming data on arrival, in order to save CPU cycles and prevent unnecessary copying of data. When enabled, the actual HTTP response body would be fully downloaded (so that the load test and all HTTP metrics for that request are still accurate), it just won't be saved in memory and passed on to the JavaScript runtime at all - the response.body attribute would be null:

import http from 'k6/http';
import { check } from "k6";

export default function () {
    const url = "http://test.loadimpact.com";
    let resp = http.get(url);
    let cssFile = resp.html().find("link[rel='stylesheet']").attr("href");

    check(http.get(`${url}/${cssFile}`, { responseType: "none" }), {
        "body was empty": (res) => res.body === null,
        "response code was 200": (res) => res.status == 200,
        "timings are present": (res) => res.timings.duration > 0,
    });
};

For convenience, there's also a new global config option that causes k6 to discard response bodies by default by switching the default responseType value to none. It can be enabled via the --discard-response-bodies CLI flag, the K6_DISCARD_RESPONSE_BODIES environment variable, or the discardResponseBodies script option:

import http from 'k6/http';
export let options = {
  discardResponseBodies: true,
};
export default function () {
  let response = http.get("http://test.loadimpact.com", { responseType: "text" });
  // ... do something with the response, but ignore the contents of static files:
  http.batch([
    "http://test.loadimpact.com/images/logo.png",
    "http://test.loadimpact.com/style.css"
  ]);
};

Thanks to @sherrman for reporting the binary handling issues that prompted the addition of the responseType option! And thanks to @ofauchon for implementing both of the discard response body options, of which the local per-request one was later transformed into the responseType=none value!

k6/http: The Response.json() method now supports selectors

The selectors are implemented with the gjson library and allow optimized lookups and basic filtering of JSON elements in HTTP responses, which could be especially useful in combination with k6 checks:

import http from "k6/http";
import { check } from "k6";

export default function () {
	let resp = http.get("https://api.spacexdata.com/v2/launches/");

	let currentYear = (new Date()).getFullYear();
	check(resp, {
		"falcon heavy": (r) => r.json("#[flight_number==55].rocket.second_stage.payloads.0.payload_id") === "Tesla Roadster",
		"no failure this year": (r) => r.json("#[launch_success==false]#.launch_year").every((y) => y < currentYear),
		"success ratio": (r) => r.json("#[launch_success==true]#").length > 10 * r.json("#[launch_success==false]#").length,
	});
}

Thanks to @AndriiChuzhynov for implementing this! (#766)

New option: disable the summary at the end of a test (#729)

A new option that disables the end-of-test summary has been added. That summary is often superfluous when k6 tests are run in a distributed execution mode, or when the generated metrics are piped to an external output like InfluxDB or Load Impact Insights. The option can be enabled with the --no-summary CLI flag or the K6_NO_SUMMARY environment variable. When both it and the and the --no-thresholds option are enabled, k6 won't store any generated metrics in-memory, making the test execution a bit more efficient.

New option: set a minimum iteration duration (#821)

You can now specify the minimum amount of time a single iteration should take via the new minIterationDuration option. It's also configurable via the --min-iteration-duration CLI flag and K6_MIN_ITERATION_DURATION environment variable. This setting only applies for full iterations, so any interrupted iterations due to ramping down of VUs from a stage or at the end of the tests can still be shorter.

UX

• Added a warning when the maximum number of VUs is more than the total number of iterations (#802)

Internals

• Cloud output: improved outlier metric detection for small batches. (#744)
• Use 20 as the the default values of the batch and batchPerHost options. They determine the maximum number of parallel requests (in total and per-host respectively) an http.batch() call will make per VU. The previous value for batch was 10 and for batchPerHost it was 0 (unlimited). We now also use their values to determine the maximum number of open idle connections in a VU. (#685)
• Due to refactoring needed for the redirect fixes, the NTLM authentication library k6 uses is changed from this to this. (#753)
• Switched the default CircleCI tests and linting to use Go 1.11.2, but we still maintain 1.10 compatibility by running all of the tests with Go 1.10.3 too. Official k6 standalone builds will also be done with Go 1.11+ from now on. (#813)
• Automated docker builds of the git master branch will now tag the resulting docker image as master as well. The latest docker tag will point to the latest stable official release, so it will be equivalent to v0.23.0 until we release the next k6 version. (#846)

Bugs fixed!

• UI: The interactive k6 login influxdb command failed to write the supplied options to the config file. (#734)
• UI: Password input is now masked in k6 login influxdb and k6 login cloud. (#734)
• Config: Environment variables can now be used to modify k6's behavior in the k6 login subcommands. (#734)
• HTTP: Binary response bodies were mangled because there was no way to avoid converting them to UTF-16 JavaScript strings. (#749)
• Config: Stages were appended instead of overwritten from upper config "tiers", and were doubled when supplied via the CLI flag. (#759)
• HAR converter: Fixed a panic due to a missing array length check. (#760)
• HTTP: http.batch() calls could panic because of a data race when the batchPerHost global option was used. (#770)
• Docker: Fixed the grafana image in the docker-compose setup. Thanks @entone and @mariolopjr! (#783)
• Config: Stages configured via the script options or environment variables couldn't be disabled via the CLI flags. (#786)
• UI: Don't report infinities and extreme speeds when tests take 0 time. Thanks @tkbky! (#790)
• HTTP: Correct metric tracking when HTTP requests are redirected. (#753)
• HAR converter: Added escaping for page IDs and names in the generated scripts. (#801)
• Setup data: Distinguish between undefined (when there is no setup() function or when it doesn't return anything) and null (when setup() explicitly returns null) values for the setup data that's passed to the default function and teardown(). (#799)
• Setup data: Prevent data races by having each VU have its own independent copy of the setup data. (#799)
• HAR converter: Support HAR files that don't have a pages array. (#806)
• Setup data: The property names of some of the internal k6 objects like http.Response weren't properly encoded when they were returned from the setup() function. (#804)
• UX: Instead of panicking on some operations in the init context, we now return an error that the given action is not supported; this includes making HTTP requests (batched or not), websockets, adding to custom metrics, making checks and groups, or initializing cookie jars. (#811)
• Cloud execution: properly send specific environment variables to the cloud option validation endpoint when running k6 cloud. Most notably this includes K6_CLOUD_PROJECT_ID. (#829)
• Cloud execution and archive running: Fixed an issue when files were imported or opened with their full paths and they were in the users' home folders. (#823)
• Script: Fixed a panic when group() is called without a callback function. (#841)

Breaking changes

• Metric names are now restricted to only allow Unicode letters or numbers, spaces, and the following special characters: ._!?/&#()<>%-. They also need to be between 1 and 128 characters. Previously practically anything was a valid metric name. (#810)