High Performance, Kubernetes Native Object Storage

Overview

MinIO Quickstart Guide

Slack Docker Pulls

MinIO

MinIO is a High Performance Object Storage released under Apache License v2.0. It is API compatible with Amazon S3 cloud storage service. Use MinIO to build high performance infrastructure for machine learning, analytics and application data workloads.

This README provides quickstart instructions on running MinIO on baremetal hardware, including Docker-based installations. For Kubernetes environments, use the MinIO Kubernetes Operator.

Docker Installation

Use the following commands to run a standalone MinIO server on a Docker container.

Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a minimum of 4 drives per MinIO server. See MinIO Erasure Code Quickstart Guide for more complete documentation.

Stable

Run the following command to run the latest stable image of MinIO on a Docker container using an ephemeral data volume:

docker run -p 9000:9000 minio/minio server /data

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

NOTE: To deploy MinIO on Docker with persistent storage, you must map local persistent directories from the host OS to the container using the docker -v option. For example, -v /mnt/data:/data maps the host OS drive at /mnt/data to /data on the Docker container.

Edge

Run the following command to run the bleeding-edge image of MinIO on a Docker container using an ephemeral data volume:

docker run -p 9000:9000 minio/minio:edge server /data

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

NOTE: To deploy MinIO on Docker with persistent storage, you must map local persistent directories from the host OS to the container using the docker -v option. For example, -v /mnt/data:/data maps the host OS drive at /mnt/data to /data on the Docker container.

macOS

Use the following commands to run a standalone MinIO server on macOS.

Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a minimum of 4 drives per MinIO server. See MinIO Erasure Code Quickstart Guide for more complete documentation.

Homebrew (recommended)

Run the following command to install the latest stable MinIO package using Homebrew. Replace /data with the path to the drive or directory in which you want MinIO to store data.

brew install minio/stable/minio
minio server /data

NOTE: If you previously installed minio using brew install minio then it is recommended that you reinstall minio from minio/stable/minio official repo instead.

brew uninstall minio
brew install minio/stable/minio

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

Binary Download

Use the following command to download and run a standalone MinIO server on macOS. Replace /data with the path to the drive or directory in which you want MinIO to store data.

wget https://dl.min.io/server/minio/release/darwin-amd64/minio
chmod +x minio
./minio server /data

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

GNU/Linux

Use the following command to run a standalone MinIO server on Linux hosts running 64-bit Intel/AMD architectures. Replace /data with the path to the drive or directory in which you want MinIO to store data.

wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
./minio server /data

Replace /data with the path to the drive or directory in which you want MinIO to store data.

The following table lists supported architectures. Replace the wget URL with the architecture for your Linux host.

Architecture URL
64-bit Intel/AMD https://dl.min.io/server/minio/release/linux-amd64/minio
64-bit ARM https://dl.min.io/server/minio/release/linux-arm64/minio
64-bit PowerPC LE (ppc64le) https://dl.min.io/server/minio/release/linux-ppc64le/minio
IBM Z-Series (S390X) https://dl.min.io/server/minio/release/linux-s390x/minio

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

NOTE: Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a minimum of 4 drives per MinIO server. See MinIO Erasure Code Quickstart Guide for more complete documentation.

Microsoft Windows

To run MinIO on 64-bit Windows hosts, download the MinIO executable from the following URL:

https://dl.min.io/server/minio/release/windows-amd64/minio.exe

Use the following command to run a standalone MinIO server on the Windows host. Replace D:\ with the path to the drive or directory in which you want MinIO to store data. You must change the terminal or powershell directory to the location of the minio.exe executable, or add the path to that directory to the system $PATH:

minio.exe server D:\

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

NOTE: Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a minimum of 4 drives per MinIO server. See MinIO Erasure Code Quickstart Guide for more complete documentation.

FreeBSD

MinIO does not provide an official FreeBSD binary. However, FreeBSD maintains an upstream release using pkg:

pkg install minio
sysrc minio_enable=yes
sysrc minio_disks=/home/user/Photos
service minio start

Install from Source

Use the following commands to compile and run a standalone MinIO server from source. Source installation is only intended for developers and advanced users. If you do not have a working Golang environment, please follow How to install Golang. Minimum version required is go1.16

GO111MODULE=on go get github.com/minio/minio

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

NOTE: Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a minimum of 4 drives per MinIO server. See MinIO Erasure Code Quickstart Guide for more complete documentation.

MinIO strongly recommends against using compiled-from-source MinIO servers for production environments.

Deployment Recommendations

Allow port access for Firewalls

By default MinIO uses the port 9000 to listen for incoming connections. If your platform blocks the port by default, you may need to enable access to the port.

ufw

For hosts with ufw enabled (Debian based distros), you can use ufw command to allow traffic to specific ports. Use below command to allow access to port 9000

ufw allow 9000

Below command enables all incoming traffic to ports ranging from 9000 to 9010.

ufw allow 9000:9010/tcp

firewall-cmd

For hosts with firewall-cmd enabled (CentOS), you can use firewall-cmd command to allow traffic to specific ports. Use below commands to allow access to port 9000

firewall-cmd --get-active-zones

This command gets the active zone(s). Now, apply port rules to the relevant zones returned above. For example if the zone is public, use

firewall-cmd --zone=public --add-port=9000/tcp --permanent

Note that permanent makes sure the rules are persistent across firewall start, restart or reload. Finally reload the firewall for changes to take effect.

firewall-cmd --reload

iptables

For hosts with iptables enabled (RHEL, CentOS, etc), you can use iptables command to enable all traffic coming to specific ports. Use below command to allow access to port 9000

iptables -A INPUT -p tcp --dport 9000 -j ACCEPT
service iptables restart

Below command enables all incoming traffic to ports ranging from 9000 to 9010.

iptables -A INPUT -p tcp --dport 9000:9010 -j ACCEPT
service iptables restart

Pre-existing data

When deployed on a single drive, MinIO server lets clients access any pre-existing data in the data directory. For example, if MinIO is started with the command minio server /mnt/data, any pre-existing data in the /mnt/data directory would be accessible to the clients.

The above statement is also valid for all gateway backends.

Test MinIO Connectivity

Test using MinIO Browser

MinIO Server comes with an embedded web based object browser. Point your web browser to http://127.0.0.1:9000 to ensure your server has started successfully.

Screenshot

Test using MinIO Client mc

mc provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff etc. It supports filesystems and Amazon S3 compatible cloud storage services. Follow the MinIO Client Quickstart Guide for further instructions.

Upgrading MinIO

MinIO server supports rolling upgrades, i.e. you can update one MinIO instance at a time in a distributed cluster. This allows upgrades with no downtime. Upgrades can be done manually by replacing the binary with the latest release and restarting all servers in a rolling fashion. However, we recommend all our users to use mc admin update from the client. This will update all the nodes in the cluster simultaneously and restart them, as shown in the following command from the MinIO client (mc):

mc admin update <minio alias, e.g., myminio>

NOTE: some releases might not allow rolling upgrades, this is always called out in the release notes and it is generally advised to read release notes before upgrading. In such a situation mc admin update is the recommended upgrading mechanism to upgrade all servers at once.

Important things to remember during MinIO upgrades

  • mc admin update will only work if the user running MinIO has write access to the parent directory where the binary is located, for example if the current binary is at /usr/local/bin/minio, you would need write access to /usr/local/bin.
  • mc admin update updates and restarts all servers simultaneously, applications would retry and continue their respective operations upon upgrade.
  • mc admin update is disabled in kubernetes/container environments, container environments provide their own mechanisms to rollout of updates.
  • In the case of federated setups mc admin update should be run against each cluster individually. Avoid updating mc to any new releases until all clusters have been successfully updated.
  • If using kes as KMS with MinIO, just replace the binary and restart kes more information about kes can be found here
  • If using Vault as KMS with MinIO, ensure you have followed the Vault upgrade procedure outlined here: https://www.vaultproject.io/docs/upgrading/index.html
  • If using etcd with MinIO for the federation, ensure you have followed the etcd upgrade procedure outlined here: https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrading-etcd.md

Explore Further

Contribute to MinIO Project

Please follow MinIO Contributor's Guide

License

Use of MinIO is governed by the Apache 2.0 License found at LICENSE.

Comments
  • I/O Timeout

    I/O Timeout

    Howdy folks,

    No matter running the latest version or not, or latest Debian (9) or not, on RAID 10, with 30 transfers and full verbose, we get:

    ERRO[0124] Unable to create object part. cause=read tcp 46.4..:443->89.114.13*.*:49244: i/o timeout source=[object-handlers.go:817:objectAPIHandlers.PutObjectPartHandler()] stack=fs-v1-helpers.go:272:fsCreateFile fs-v1-multipart.go:523:fsObjects.PutObjectPart :339:(*fsObjects).PutObjectPart object-handlers.go:814:objectAPIHandlers.PutObjectPartHandler api-router.go:46:(objectAPIHandlers).PutObjectPartHandler-fm

    Any hint?

    Disks are fine, plenty of IO available. And these are mostly EPS files, jpg, etc, only 63GB total. I got this very same error in two servers.

    priority: medium 
    opened by xmovu 89
  • Error : hash does not match

    Error : hash does not match

    after updating minio to 2021-05-27T22:06:31Z we got for alot of files such a message in the console

    minio04 API: SYSTEM() minio04 Time: 10:27:54 UTC 06/05/2021 minio04 DeploymentID: 7c507082-fe65-439a-9391-3b44b1b1d2ed minio04 Error: Disk: http://minio01:9000/drive6 -> alwaqiyah/Cong_Sweden_Fiter1441SD.mp4/aed31203-8a34-4a28-92dc-063bfe0dc34c/part.1 - content hash does not match - expected c5e2c21880064196b4c8c21bad1b08452ba8662ae32408d92343086a34412322, got 73462c13ceab3d7d4e1849af26b94fa716ad868d9080344999a61e6ab9bc5510 (*errors.errorString) minio04 2: cmd/bitrot-streaming.go:179:cmd.(*streamingBitrotReader).ReadAt() minio04 1: cmd/erasure-decode.go:163:cmd.(*parallelReader).Read.func1()

    minio04 API: SYSTEM() minio04 Time: 10:27:54 UTC 06/05/2021 minio04 DeploymentID: 7c507082-fe65-439a-9391-3b44b1b1d2ed minio04 Error: Disk: http://minio03:9000/drive6 -> alwaqiyah/Cong_Sweden_Fiter1441SD.mp4/aed31203-8a34-4a28-92dc-063bfe0dc34c/part.1 - content hash does not match - expected 6319f76775b13b6f23495b4e53374f7629ce37ce27ccb971ae42bed55efe5771, got e271da0633bf2c805eb576bf3b7379013b27d8621ad066c015de7477cde0ced0 (*errors.errorString) minio04 2: cmd/bitrot-streaming.go:179:cmd.(*streamingBitrotReader).ReadAt() minio04 1: cmd/erasure-decode.go:163:cmd.(*parallelReader).Read.func1()

    Current Behavior

    in the browser we got something like <Error><Code>SlowDown</Code><Message>Resource requested is unreadable, please reduce your request rate</Message><Key>Cong_Sweden_Fiter1441SD.mp4</Key><BucketName>alwaqiyah</BucketName><Resource>/Cong_Sweden_Fiter1441SD.mp4</Resource><RequestId>1685A99F3532F870</RequestId><HostId>7c507082-fe65-439a-9391-3b44b1b1d2ed</HostId></Error>

    server enviroment

    • Version 2021-05-27T22:06:31Z
    • Server setup and configuration: 4 nodes CPU : Intel(R) Xeon(R) CPU D-1541 @ 2.10GHz 8C /16T MEMORY: 32G Disk: 6 disk 11T per disk
    • Operating System and version (uname -a): Linux minio04 5.4.0-73-generic #82-Ubuntu SMP Wed Apr 14 17:39:42 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
    community 
    opened by mazataza 74
  • Minio server stops in every few hours

    Minio server stops in every few hours

    Expected Behavior

    Expected behavior is for Server to remain up till it is not stopped.

    Current Behavior

    Server stops(process dies) in every few hours.

    Steps to Reproduce (for bugs)

    1. docker pull minio/minio:edge
    2. docker run -p 9000:9000 minio/minio:edge server /export
    3. Wait for few hours and the Server would go down

    Context

    Because of this issue, I need to restart the Server in every few hours. Then I restart it and a new public and private key is generated. Then I need to change the keys in my code and redeploy it. A new Server instance is created every time I restart it, which essentially means that all my files, stored in previous Server instance, get lost.

    Your Environment

    • Server type and version: Latest from Master branch
    • Operating System and version: centos-release-7-3.1611.el7.centos.x86_64
    community 
    opened by puneetpandey37 59
  • Minio cluster fails to start/sync

    Minio cluster fails to start/sync "Waiting for a minimum of 2 disks to come online"

    Expected Behavior

    I expect it to start or at least provide log data as to what is wrong.

    Current Behavior

    All 4 nodes show the same data in the log.

    Waiting for a minimum of 2 disks to come online (elapsed 16m31s) Waiting for a minimum of 2 disks to come online (elapsed 16m32s) Waiting for a minimum of 2 disks to come online (elapsed 16m33s) Waiting for a minimum of 2 disks to come online (elapsed 16m34s) Waiting for a minimum of 2 disks to come online (elapsed 16m35s)

    If I log into a container I can ping the other containers and a curl to the http://minio#:9000/data produces the following

    <Error><Code>XMinioServerNotInitialized</Code><Message>Server not initialized, please try again.</Message><BucketName>data</BucketName><Resource>/data</Resource><RequestId>15F7158280B18599</RequestId><HostId></HostId></Error>/

    Steps to Reproduce (for bugs)

    Note sure how to reproduce but it happened when my docker swarm failed and I had to recreate it. While digging around in the config for minio, stored in the volume I noticed that recreating the stack updated to a newer minio so the old version was RELEASE.2019-09-26T19-42-35Z, while the new one is RELEASE.2020-02-20T22-51-23Z.

    Note: This had run for months without issue.

    Your Environment

    • Version used (minio version): RELEASE.2020-02-20T22-51-23Z
    • Environment name and version (e.g. nginx 1.9.1): Docker 19.03.6
    • Server type and version: 20 VM Docker Swarm
    • Operating System and version (uname -a): (docker nodes are) CentOS Linux release 7.7.1908 (Core)

    I've been working on this for about a week, digging through posts to see if I can find a way to get it back up and have not found anything. So I thought I would ask here.

    For Reference, this is my stack.yaml

    version: "3.2"
    services:
      minio1:
        image: minio/minio
        volumes:
          - data2:/data
        networks:
          - traefik_default
          - minio
        deploy:
           labels:
              - "traefik.frontend.rule=Host:minio-c.docker.mydomain.com"
              - "traefik.port=9000"
              - "traefik.enable=true"     
        environment:
          MINIO_ACCESS_KEY: <MY ACCESS KEY>
          MINIO_SECRET_KEY: <MY SECRET KEY>
        command: server  http://minio1:9000/data http://minio4:9000/data http://minio2:9000/data http://minio3:9000/data
    
      minio2:
        image: minio/minio
        volumes:
          - data3:/data
        networks:
          - traefik_default
          - minio
        deploy:
           labels:
              - "traefik.frontend.rule=Host:minio-c.docker.mydomain.com"
              - "traefik.port=9000"
              - "traefik.enable=true"     
        environment:
          MINIO_ACCESS_KEY: <MY ACCESS KEY>
          MINIO_SECRET_KEY: <MY SECRET KEY>
        command: server http://minio2:9000/data http://minio4:9000/data http://minio1:9000/data http://minio3:9000/data
    
      minio3:
        image: minio/minio
        volumes:
          - data4:/data
        networks:
          - traefik_default
          - minio
        deploy:
           labels:
              - "traefik.frontend.rule=Host:minio-c.docker.mydomain.com"
              - "traefik.port=9000"
              - "traefik.enable=true"     
        environment:
          MINIO_ACCESS_KEY: <MY ACCESS KEY>
          MINIO_SECRET_KEY: <MY SECRET KEY>
        command: server http://minio3:9000/data http://minio4:9000/data http://minio1:9000/data http://minio2:9000/data 
    
      minio4:
        image: minio/minio
        volumes:
          - data5:/data
        networks:
          - traefik_default
          - minio
        deploy:
           labels:
              - "traefik.frontend.rule=Host:minio-c.docker.mydomain.com"
              - "traefik.port=9000"
              - "traefik.enable=true"     
        environment:
          MINIO_ACCESS_KEY: <MY ACCESS KEY>
          MINIO_SECRET_KEY: <MY SECRET KEY>
        command: server http://minio4:9000/data http://minio1:9000/data http://minio2:9000/data http://minio3:9000/data
        
    <Removed Volumes>
    <Removed Networks>
    
    
    working as intended community 
    opened by Aphris-Karu 58
  • Implement S3 Gateway to third party cloud storage providers.

    Implement S3 Gateway to third party cloud storage providers.

    Description

    Currently supported backend is Azure Blob Storage.

    export MINIO_ACCESS_KEY=azureaccountname
    export MINIO_SECRET_KEY=azureaccountkey
    minio gateway azure
    

    Motivation and Context

    Minio gateway adds Amazon S3 compatibility to third party cloud storage providers.

    How Has This Been Tested?

    Manually and by rest of the @minio/core

    Types of changes

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [x] New feature (non-breaking change which adds functionality)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)

    Checklist:

    • [x] My code follows the code style of this project.
    • [x] My change requires a change to the documentation.
    • [x] I have updated the documentation accordingly.
    • [x] I have added tests to cover my changes.
    • [x] All new and existing tests passed.
    opened by krishnasrinivas 57
  • WebUI/listObjects() show/return object. Download/getObject() return empty file / cause error

    WebUI/listObjects() show/return object. Download/getObject() return empty file / cause error "NoSuchKey"

    The WebUI displays a file with a plausible size of 740B. Screenshot 2022-06-05 144902

    Downloading this file via WebUI results in an empty file: Screenshot 2022-06-05 145130

    API access by way of listObjects() returns an Iterator<Result> results, which includes the above mentioned objectName.

    Calling getObject() with the objectName from above causes an ErrorResponseException with an errorResponseCode "NoSuchKey".

    Expected Behavior

    Parts of Minio seem to think the file exists, others don't. A displayed / listed objectName should be retrievable.

    Current Behavior

    See description.

    Trace of a download attempt by WebUI: Screenshot 2022-06-05 150522

    Context

    Usecase is a regular low intensity consistency check with a cassandra db. We traverse all Minio objects and check if cassandra is up-to-date.

    Regression

    na

    Your Environment

    MinIO VERSION 2022-04-16T04:26:02Z 4 dedicated Rasp-4B with 8GB each with dedicated 2TB SSD. Linux rasp-3 5.15.32-v8+ #1538 SMP PREEMPT Thu Mar 31 19:40:39 BST 2022 aarch64 GNU/Linux

    This has been working like a charm for about a year with > 16 Mio object and 2 TB data.

    community priority: medium 
    opened by RAlfoeldi 55
  • Hadoop 3.3 Compatibility issue with single drive mode (minio server /data)

    Hadoop 3.3 Compatibility issue with single drive mode (minio server /data)

    MinIO works with Hadoop 3.2, but not with Hadoop 3.3.

    Current Behavior

    Repro code:

    val conf = new Configuration()
    conf.set("fs.s3a.endpoint", "http://127.0.0.1:9000")
    conf.set("fs.s3a.path.style.access", "true")
    conf.set("fs.s3a.access.key", "user_access_key")
    conf.set("fs.s3a.secret.key", "password")
    
    val path = new Path("s3a://comcast-test")
    val fs = path.getFileSystem(conf)
    
    fs.mkdirs(new Path("/testdelta/_delta_log"))
    fs.getFileStatus(new Path("/testdelta/_delta_log"))
    

    Fails with FileNotFoundException. The same code works in real S3. It also works in Hadoop 3.2. Only fails on 3.3 and newer Hadoop branches.

    Possible Solution

    This works in Hadoop 3.2 because of this infamous "Is this necessary?" block of code https://github.com/apache/hadoop/blob/branch-3.2.0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L2204-L2223

    that was removed in Hadoop 3.3 - https://github.com/apache/hadoop/blob/branch-3.3.0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L2179

    and this causes the regression

    Steps to Reproduce (for bugs)

    See code above.

    Context

    Some applications need to create subdirectories before they can write to Minio, so this affects

    Regression

    Yes, this is a regression.

    Your Environment

    • Subnet ticket 3439
    community fixed 
    opened by Tagar 55
  • AWS iOS SDK working with minio

    AWS iOS SDK working with minio

    Error messgae

    SignatureDoesNotMatchThe request signature we calculated does not match the signature you provided. Check your key and signing method./images/test.jpg3L1373L137 Upload failed with error: (The operation couldn’t be completed. (com.amazonaws.AWSServiceErrorDomain error 3.))

    Here is how I configure

    let accessKey = "xxxxxxx"
    let secretKey = "xxxxxxx"       
    let credentialsProvider = AWSStaticCredentialsProvider(accessKey: accessKey, secretKey: secretKey)
    let configuration = AWSServiceConfiguration(region: AWSRegionType.USEast1, endpoint: AWSEndpoint(region: .USEast1, service: .APIGateway, url: URL( string:"http://xxxx.com:9000")),credentialsProvider: credentialsProvider)
    AWSServiceManager.default().defaultServiceConfiguration = configuration
    

    Possible Solution

    refer to this issue, https://github.com/minio/mc/issues/1707 maybe it caused by region issue?

    Your Environment

    • Version used:

      • official docker latest
      • awss3 lib used: pod 'AWSS3', '~> 2.5'
    community priority: high 
    opened by atom2ueki 54
  • ecosystem: Validate all spark and hadoop supported s3 connectors

    ecosystem: Validate all spark and hadoop supported s3 connectors

    Expected behaviour

    I want to use spark-shell (v2.0.1) to connect to minio local to read input data and write back output data.

    Actual behaviour

    Attempting a simple read of a text file (json in this case) results in the error:

    com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 400, AWS Service: Amazon S3, AWS Request ID: null, AWS Error Code: null, AWS Error Message: Bad Request
    

    (full stack will be below)

    Steps to reproduce the behaviour

    Install Oracle Java 8 (1.8.0_71) Install Scala 2.11.8 Minio installed by Home Brew Install Spark 2.0.1

    Start the minio server local, log into the browser UI, create a bucket named data and upload a small json file to it (e.g. bacon_tiny.json).

    { "bacon": {
        "isCrispy": true,
        "isDelicious": true,
        "isVegan": false
        }
    }
    

    Relevant bits of spark-env.sh (all other bits left to default):

    HADOOP_CONF_DIR=/Users/bkarels/hadoop/hadoop-2.7.3/etc/hadoop
    SPARK_LOCAL_IP="127.0.0.1"
    SPARK_MASTER_HOST="localhost"
    SPARK_MASTER_WEBUI_PORT=8080
    SPARK_MASTER_PORT=7077
    SPARK_DAEMON_JAVA_OPTS="-Djava.net.preferIPv4Stack=true -Dcom.amazonaws.services.s3.enableV4=true"
    

    core-site.xml (from ~/hadoop/hadoop-2.7.3/etc/hadoop):

    <?xml version="1.0" encoding="UTF-8"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    
    <configuration>
      <property>
        <name>fs.s3a.endpoint</name>
        <value>http://127.0.0.1:9000</value>
      </property>
    
      <property>
        <name>fs.s3a.access.key</name>
        <description>AWS access key ID.</description>
        <value>*************************</value>
      </property>
    
      <property>
        <name>fs.s3a.secret.key</name>
        <description>AWS secret key.</description>
        <value>*********************************************</value>
      </property>
    
    </configuration>
    

    Starting the spark-shell

    $ ./bin/spark-shell --master local[4] --jars "./bin/hadoop-aws-2.7.1.jar,./bin/aws-java-sdk-1.7.4.jar"
    scala> val bacon = sc.textFile("s3a://data/bacon_tiny.json").first
    
    com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 400, AWS Service: Amazon S3, AWS Request ID: null, AWS Error Code: null, AWS Error Message: Bad Request
      at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:798)
      at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:421)
      at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232)
      at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528)
      at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1031)
      at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:994)
      at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:297)
      at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
      at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
      at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
      at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
      at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
      at org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:258)
      at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:229)
      at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
      at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:199)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:248)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:246)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.rdd.RDD.partitions(RDD.scala:246)
      at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:248)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:246)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.rdd.RDD.partitions(RDD.scala:246)
      at org.apache.spark.rdd.RDD$$anonfun$take$1.apply(RDD.scala:1303)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
      at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
      at org.apache.spark.rdd.RDD.take(RDD.scala:1298)
      at org.apache.spark.rdd.RDD$$anonfun$first$1.apply(RDD.scala:1338)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
      at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
      at org.apache.spark.rdd.RDD.first(RDD.scala:1337)
      ... 48 elided
    

    All else being equal, I can adjust core-site.xml to point to an actual s3 instance and things work which is what has me thinking this might be a Minio issue.

    Minio version

    Version: 2016-09-11T17:42:18Z
    Release-Tag: RELEASE.2016-09-11T17-42-18Z
    Commit-ID: 85e2d886bcb005d49f3876d6849a2b5a55e03cd3
    

    System information

    Running on OSX 10.11.6
    MacBook Pro (Retina, 15-inch, Mid 2015)
    2.5 GHz Intel Core i7
    16 GB 1600 MHz DDR3
    
    community fixed priority: medium 
    opened by bradkarels 54
  • ARM64 : Minio server crashes randomly

    ARM64 : Minio server crashes randomly

    Minio server is terminating randomly on ARM64 platform. Built the latest code on ARM64 platform and used for testing.The deployment type is "distributed mode". In a week testing on 6 nodes, 3 servers crashed. The load was moderate when these crashes happened.

    Log messages ::

    goroutine 338370336 [IO wait]: runtime.gopark(0x1821768, 0xfffe25850a98, 0x4001841b02, 0x5) /usr/local/go/src/runtime/proc.go:301 +0xf0 fp=0x40042fc9c0 sp=0x40042fc9a0 pc=0x43cf50 runtime.netpollblock(0xfffe25850a70, 0x72, 0x409266a000) /usr/local/go/src/runtime/netpoll.go:389 +0xa4 fp=0x40042fca00 sp=0x40042fc9c0 pc=0x4385f4 internal/poll.runtime_pollWait(0xfffe25850a70, 0x72, 0xffffffffffffffff) /usr/local/go/src/runtime/netpoll.go:182 +0x48 fp=0x40042fca30 sp=0x40042fca00 pc=0x437b48 internal/poll.(*pollDesc).wait(0x408b030818, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0xa0 fp=0x40042fca60 sp=0x40042fca30 pc=0x493fc0 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(*FD).Read(0x408b030800, 0x409266a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) /usr/local/go/src/internal/poll/fd_unix.go:169 +0x170 fp=0x40042fcac0 sp=0x40042fca60 pc=0x494e40 net.(*netFD).Read(0x408b030800, 0x409266a000, 0x1000, 0x1000, 0x40042fcb78, 0x413008, 0x40042fcb60) /usr/local/go/src/net/fd_unix.go:202 +0x44 fp=0x40042fcb20 sp=0x40042fcac0 pc=0x598c14 net.(*conn).Read(0x4031f50158, 0x409266a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) /usr/local/go/src/net/net.go:177 +0x5c fp=0x40042fcb80 sp=0x40042fcb20 pc=0x5ab42c net/http.(*persistConn).Read(0x408dc807e0, 0x409266a000, 0x1000, 0x1000, 0x40042fcc68, 0x6c2f30, 0x405fa8c840) /usr/local/go/src/net/http/transport.go:1524 +0x60 fp=0x40042fcc00 sp=0x40042fcb80 pc=0x6c1da0 bufio.(*Reader).fill(0x4004105a40) /usr/local/go/src/bufio/bufio.go:100 +0x100 fp=0x40042fcc50 sp=0x40042fcc00 pc=0x4b0000 bufio.(*Reader).Peek(0x4004105a40, 0x1, 0x0, 0x0, 0x1, 0x400d6c5600, 0x0) /usr/local/go/src/bufio/bufio.go:138 +0x40 fp=0x40042fcc70 sp=0x40042fcc50 pc=0x4b0180 net/http.(*persistConn).readLoop(0x408dc807e0) /usr/local/go/src/net/http/transport.go:1677 +0x16c fp=0x40042fcfd0 sp=0x40042fcc70 pc=0x6c282c runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1128 +0x4 fp=0x40042fcfd0 sp=0x40042fcfd0 pc=0x46aa74 created by net/http.(*Transport).dialConn /usr/local/go/src/net/http/transport

    Expected Behavior

    Minio server shouldn't terminate.

    Current Behavior

    Minio server terminates.

    Your Environment

    • Version used (minio version): Minio (built from source on ARM64 platform) Version: DEVELOPMENT.GOGET Release-Tag: DEVELOPMENT.GOGET Commit-ID: DEVELOPMENT.GOGET

    • Operating System and version (uname -a): aarch64 aarch64 aarch64 GNU/Linux

    working as intended community priority: medium 
    opened by kattakarthik 53
  • Scheduled removal of MinIO Gateway for GCS, Azure, HDFS

    Scheduled removal of MinIO Gateway for GCS, Azure, HDFS

    MinIO Gateway will be removed by June 1st, 2022 from the MinIO repository:

    Community Users

    • Please migrate your MinIO Gateway deployments from Azure, GCS, HDFS to MinIO Distributed Setups

    • MinIO S3 Gateway will be renamed as "minio edge" and will only support MinIO Backends to extend the functionality of supporting remote credentials etc locally as "read-only" for authentication and policy management.

    • Newer MinIO NAS/Single drive setups will move to single data and 0 parity mode (that will re-purpose the erasure-coded backend used for distributed setups but with 0 parity). This would allow for distributed setup features to be available for single drive deployments as well such as

      • Versioning
      • ILM
      • Replication and more...
    • Existing setups for NAS/Single drive setups will work as-is nothing changes.

    Paid Users

    All existing paid customers will be supported as per their LTS support contract. If there are bugs they will be fixed and backported fixes will be provided. No new features will be implemented for Gateway implementations.

    community fixed priority: high 
    opened by harshavardhana 50
  • fix: de-duplicate conflicting object names on namespace

    fix: de-duplicate conflicting object names on namespace

    Description

    fix: de-duplicate conflicting object names on namespace

    Motivation and Context

    In a hypothetical scenario

    • foo-1 (object)
    • foo-1/ (directory)

    foo-1 must take priority and foo-1/ must be be de-duplicated with the actual object.

    This is mainly to avoid cases where we end up listing both foo-1 and foo-1/ together. This can happen when

    • foo-1 is an object
    • foo-1/ is just a folder elsewhere on another erasure set.

    How to test this PR?

    We need to hand construct this scenario but it is possible to reach this situation in the wild here is how you do it, have atleast "two" erasure sets

    • create an object foo-1 and let it stay on one erasure set, now you go ahead
    • mkdir foo-1/ on the second erasure set - this will lead to foo-1/ getting listed incorrectly.

    Types of changes

    • [x] Bug fix (non-breaking change which fixes an issue)
    • [ ] New feature (non-breaking change which adds functionality)
    • [ ] Optimization (provides speedup with no functional changes)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)

    Checklist:

    • [ ] Fixes a regression (If yes, please add commit-id or PR # here)
    • [ ] Unit tests added/updated
    • [ ] Internal documentation updated
    • [ ] Create a documentation update request here
    opened by harshavardhana 1
  • Re-load config after reset

    Re-load config after reset

    Description

    Currently, after resetting a config for a given sub-system, it doesn't return the default config on running mc admin config get

    e.g.

    > mc admin config get myminio subnet
    subnet
    

    instead of

    > mc admin config get myminio subnet
    subnet api_key= license= proxy=
    

    It starts returning the correct output after server restart. Fixed by ensuring that the config is re-read after reset so that the default values of sub-systems get set.

    Motivation and Context

    Fix incorrect output of mc admin config get after running mc admin config reset

    How to test this PR?

    • Reset config for a sub-system e.g. mc admin config reset subnet
    • Get the same config e.g. mc admin config get subnet
    • Verify that it contains the default key-values of that subsystem e.g. subnet api_key= license= proxy=

    Types of changes

    • [x] Bug fix (non-breaking change which fixes an issue)
    • [ ] New feature (non-breaking change which adds functionality)
    • [ ] Optimization (provides speedup with no functional changes)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)

    Checklist:

    • [x] Fixes a regression https://github.com/minio/minio/pull/15465
    • [ ] Unit tests added/updated
    • [ ] Internal documentation updated
    • [ ] Create a documentation update request here
    opened by anjalshireesh 1
  • [WIP] Add MoreRecent to FileInfo

    [WIP] Add MoreRecent to FileInfo

    Description

    This field enables ILM subsystem to filter out objects expired by NewerNoncurrentVersions from GET/HEAD/ListObject APIs.

    Pending

    • [ ] Verify that filtering of objects expired due to NewerNoncurrentVersions work as expected in GET/HEAD/ListObject APIs

    Motivation and Context

    How to test this PR?

    Types of changes

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [ ] New feature (non-breaking change which adds functionality)
    • [ ] Optimization (provides speedup with no functional changes)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)

    Checklist:

    • [ ] Fixes a regression (If yes, please add commit-id or PR # here)
    • [ ] Unit tests added/updated
    • [ ] Internal documentation updated
    • [ ] Create a documentation update request here
    opened by krisis 2
  • Fix a race found in unit tests

    Fix a race found in unit tests

    This is probably introduced after the RPC tcp stats but it is not clear how it is related yet

    === RUN   TestAPIListObjectPartsHandlerPreSign
    WARNING: DATA RACE
    Read at 0x00c0364bf4d0 by goroutine 1044:
      runtime.racereadrange()
          <autogenerated>:1 +0x1b
      runtime.deferreturn()
          /opt/hostedtoolcache/go/1.18.6/x64/src/runtime/panic.go:436 +0x32
      github.com/minio/minio/cmd.newlockRESTClient.func1()
          /home/runner/work/minio/minio/cmd/lock-rest-client.go:163 +0x104
      github.com/minio/minio/internal/rest.(*Client).MarkOffline.func1()
          /home/runner/work/minio/minio/internal/rest/client.go:359 +0xb5
    
    Previous write at 0x00c0364bf4d0 by goroutine 3077:
      github.com/minio/minio/internal/rest.setupReqStatsUpdate.func1()
          /home/runner/work/minio/minio/internal/rest/rpc-stats.go:61 +0x4d
      net.(*sysDialer).dialSingle()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/dial.go:573 +0x173
      net.(*sysDialer).dialSerial()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/dial.go:551 +0x471
      net.(*Dialer).DialContext()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/dial.go:428 +0xabb
      github.com/minio/minio/internal/http.NewInternodeDialContext.func1()
          /home/runner/work/minio/minio/internal/http/dial_linux.go:68 +0xe5
      github.com/minio/minio/internal/http.DialContextWithDNSCache.func1()
          /home/runner/work/minio/minio/internal/http/dial_dnscache.go:63 +0x343
      net/http.(*Transport).dial()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/http/transport.go:1169 +0x269
      net/http.(*Transport).dialConn()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/http/transport.go:1607 +0xd57
      net/http.(*Transport).dialConnFor()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/http/transport.go:1449 +0x13a
      net/http.(*Transport).queueForDial.func1()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/http/transport.go:1418 +0x47
    
    Goroutine 1044 (running) created at:
      github.com/minio/minio/internal/rest.(*Client).MarkOffline()
          /home/runner/work/minio/minio/internal/rest/client.go:354 +0x364
      github.com/minio/minio/internal/rest.(*Client).Call()
          /home/runner/work/minio/minio/internal/rest/client.go:252 +0x5a4
      github.com/minio/minio/cmd.(*lockRESTClient).callWithContext()
          /home/runner/work/minio/minio/cmd/lock-rest-client.go:65 +0x134
      github.com/minio/minio/cmd.(*lockRESTClient).restCall()
          /home/runner/work/minio/minio/cmd/lock-rest-client.go:97 +0x21e
      github.com/minio/minio/cmd.(*lockRESTClient).RLock()
          /home/runner/work/minio/minio/cmd/lock-rest-client.go:111 +0x19b
      github.com/minio/minio/cmd.TestLockRESTlient()
          /home/runner/work/minio/minio/cmd/lock-rest-client_test.go:40 +0x15a
      testing.tRunner()
          /opt/hostedtoolcache/go/1.18.6/x64/src/testing/testing.go:1439 +0x213
      testing.(*T).Run.func1()
          /opt/hostedtoolcache/go/1.18.6/x64/src/testing/testing.go:1486 +0x47
    
    Goroutine 3077 (running) created at:
      net/http.(*Transport).queueForDial()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/http/transport.go:1418 +0x172
      net/http.(*Transport).getConn()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/http/transport.go:1372 +0x893
      net/http.(*Transport).roundTrip()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/http/transport.go:581 +0xcc4
      net/http.(*Transport).RoundTrip()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/http/roundtrip.go:17 +0x36
      net/http.send()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/http/client.go:252 +0x93e
      net/http.(*Client).send()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/http/client.go:176 +0x157
      net/http.(*Client).do()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/http/client.go:725 +0x1064
      net/http.(*Client).Do()
          /opt/hostedtoolcache/go/1.18.6/x64/src/net/http/client.go:593 +0x4f1
      github.com/minio/minio/internal/rest.(*Client).Call()
          /home/runner/work/minio/minio/internal/rest/client.go:246 +0x4c8
      github.com/minio/minio/cmd.newlockRESTClient.func1()
          /home/runner/work/minio/minio/cmd/lock-rest-client.go:163 +0x104
      github.com/minio/minio/internal/rest.(*Client).MarkOffline.func1()
          /home/runner/work/minio/minio/internal/rest/client.go:359 +0xb5
    ==================
        testing.go:1312: race detected during execution of test
    --- FAIL: TestAPIListObjectPartsHandlerPreSign (91.89s)
    
    
    community triage 
    opened by vadmeste 9
  • MINIO_AUDIT_KAFKA envs not work after RELEASE.2022-02-12T00-51-25Z

    MINIO_AUDIT_KAFKA envs not work after RELEASE.2022-02-12T00-51-25Z

    We use MINIO_AUDIT_KAFKA environment variables to write the audit log to the Kafka cluster. After we upgraded the minio cluster, we find audit log configurations did not work. After I tested it locally, it works at version RELEASE.2022-02-07T08-17-33Z and fails at version RELEASE.2022-02-12T00-51-25Z.

    Expected Behavior

    ❯ docker run -p 9000:9000 -p 9001:9001 -e MINIO_AUDIT_KAFKA_ENABLE=on -e MINIO_AUDIT_KAFKA_BROKERS=localhost:1234 -e MINIO_AUDIT_KAFKA_TOPIC=sad quay.io/minio/minio:RELEASE.2022-02-07T08-17-33Z server /data --console-address ":9001"
    
    API: SYSTEM()
    Time: 05:13:35 UTC 09/28/2022
    DeploymentID: 6e8a1ba3-f712-4d21-9451-ae6f57fcbdca
    Error: Unable to initialize server audit Kafka target: dial tcp 127.0.0.1:1234: connect: connection refused (*fmt.wrapError)
           7: cmd/config-current.go:582:cmd.lookupConfigs()
           6: cmd/config-current.go:778:cmd.loadConfig()
           5: cmd/config.go:235:cmd.initConfig()
           4: cmd/config.go:195:cmd.(*ConfigSys).Init()
           3: cmd/server-main.go:417:cmd.initConfigSubsystem()
           2: cmd/server-main.go:352:cmd.initServer()
           1: cmd/server-main.go:549:cmd.serverMain()
    WARNING: Detected default credentials 'minioadmin:minioadmin', we recommend that you change these values with 'MINIO_ROOT_USER' and 'MINIO_ROOT_PASSWORD' environment variables
    API: http://172.17.0.2:9000  http://127.0.0.1:9000
    
    Console: http://172.17.0.2:9001 http://127.0.0.1:9001
    
    Documentation: https://docs.min.io
    

    Current Behavior

    ❯ docker run -p 9000:9000 -p 9001:9001 -e MINIO_AUDIT_KAFKA_ENABLE=on -e MINIO_AUDIT_KAFKA_BROKERS=localhost:1234 -e MINIO_AUDIT_KAFKA_TOPIC=sad quay.io/minio/minio:RELEASE.2022-02-12T00-51-25Z server /data --console-address ":9001"
    API: http://172.17.0.2:9000  http://127.0.0.1:9000
    
    Console: http://172.17.0.2:9001 http://127.0.0.1:9001
    
    Documentation: https://docs.min.io
    

    Possible Solution

    make MINIO_AUDIT_KAFKA environment variable take effect

    Steps to Reproduce (for bugs)

    setup minio as the command above

    Context

    We want to enable audit logging via a Kafka topic and have been able to do so. We want to keep the configurations after upgrading instead of finding another way to collect the audit log.

    Your Environment

    • Version used (minio --version): RELEASE.2022-02-12T00-51-25Z
    • Server setup and configuration: docker container on a physical machine
    • Operating System and version (uname -a): Linux U-PF2X4A51 5.4.0-96-generic #109~18.04.1-Ubuntu SMP Thu Jan 13 15:06:26 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
    community triage 
    opened by huzhenwei7z 2
  • mc mirror fails with umlaut (ä,ü,ö) and whitespace in filename

    mc mirror fails with umlaut (ä,ü,ö) and whitespace in filename

    NOTE

    If this case is urgent, please subscribe to Subnet so that our 24/7 support team may help you faster.

    mc mirror fails when mirroring two deployments always on the same file, and a restart will result in the same error, reeor logs however do not make much sense to me.

    Logs from "mc mirror old_ls local_ls --debug" (only for the error):

    mc: Failed to copy https://URL/folder/subfolder/1002253-Bügelkurier 6:20.png. Put "http://172.17.0.2:9000/folder/subfolder/1002253-B%C3%BCgelkurier%206%3A20.png": http: ContentLength=1336204 with Body length 0 (3) mirror-main.go:503 cmd.(*mirrorJob).monitorMirrorStatus(..) Tags: [https://URL/folder/subfolder/1002253-Bügelkurier 6:20.png] (2) common-methods.go:611 cmd.uploadSourceToTargetURL(..) Tags: [https://URL/folder/subfolder/1002253-Bügelkurier 6:20.png] (1) common-methods.go:341 cmd.putTargetStream(..) Tags: [local_ls, http://172.17.0.2:9000/folder/subfolder/1002253-Bügelkurier 6:20.png] (0) client-s3.go:1112 cmd.(*S3Client).Put(..) Commit:c2c2ab4299bb | Release-Tag:RELEASE.2022-08-11T00-30-48Z | Host:Inkoginkgo | OS:linux | Arch:amd64 | Lang:go1.18.5 | Mem:11 MB/38 MB | Heap:11 MB/27 MB ...Hutchison_Drei_April-Mai2020.png: 33.08 MiB / 33.08 MiB ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 1.47 MiB/s mc: Unable to list comparison retrying.. Get "https://URL/folder/?continuation-token=MDFfcmF3LzEwMzAyNTMtMDVfMDRfcHJvZm90b19sZWFzaW5nLnBuZw%3D%3D&delimiter=&encoding-type=url&fetch-owner=true&list-type=2&prefix=": context canceled (1) difference.go:192 cmd.differenceInternal(..) Tags: [https://URL/, http://172.17.0.2:9000/] (0) client-s3.go:2245 cmd.(*S3Client).listRecursiveInRoutine(..) Release-Tag:RELEASE.2022-08-11T00-30-48Z | Commit:c2c2ab4299bb | Host:Inkoginkgo | OS:linux | Arch:amd64 | Lang:go1.18.5 | Mem:11 MB/38 MB | Heap:11 MB/27 MB

    Where you can see that the file with "Ü" poses a problem, whereas a file without an umlaut/whitespace poses no problem.

    Expected Behavior

    mc mirror should just mirror all buckets and files between two deployments

    Current Behavior

    mc mirror stops with "failed to copy" and "unable to list comparison retrying" errors

    Possible Solution

    Currently trying to manually download the problematic file and put in on the ne deployment, this is however difficult cause the folder has so many items it causes the minio webinterface to timeout

    Steps to Reproduce (for bugs)

    1. mc mirror old_minio new_minio

    Context

    Can´t backup server.

    Your Environment

    • Version used (minio --version): Cluster - MinIO VERSION 2021-12-29T06:49:06Z, Local - MinIO VERSION 2022-08-26T19:53:15Z

    Thanks for help.

    community triage 
    opened by schoenfbe 27
Releases(RELEASE.2022-09-25T15-44-53Z)
  • RELEASE.2022-09-25T15-44-53Z(Sep 26, 2022)

    What's Changed

    • fix: parse and save retainUntilDate in correct time format by @harshavardhana in https://github.com/minio/minio/pull/15741
    • validate correct ETag for the parts sent during CompleteMultipart by @harshavardhana in https://github.com/minio/minio/pull/15751
    • Relax retention date enforcement on replication target by @poornas in https://github.com/minio/minio/pull/15752
    • Add auto configuration of replication workers by @poornas in https://github.com/minio/minio/pull/15636
    • remove unused package internal/smart by @harshavardhana in https://github.com/minio/minio/pull/15758

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-09-22T18-57-27Z...RELEASE.2022-09-25T15-44-53Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-09-22T18-57-27Z(Sep 23, 2022)

    What's Changed

    • Helm conditions by @mersl in https://github.com/minio/minio/pull/15599
    • helm: default additional labels/annotations to object by @kasuboski in https://github.com/minio/minio/pull/15698
    • enable cross compile for openbsd/amd64 by @harshavardhana in https://github.com/minio/minio/pull/15701
    • Implement KMS methods for keys, policies and identities by @reivaj05 in https://github.com/minio/minio/pull/15673
    • Remove deprecated io/ioutil by @klauspost in https://github.com/minio/minio/pull/15707
    • helm: specify service account for the jobs by @cowjen01 in https://github.com/minio/minio/pull/15706
    • site replication: clear config if remove --all specified by @poornas in https://github.com/minio/minio/pull/15716
    • Fix cancellation leaks by @jiuker in https://github.com/minio/minio/pull/15721
    • persist the non-default creds in config by @harshavardhana in https://github.com/minio/minio/pull/15711
    • Fix ignoring odirect in case of erasure single disk by @vadmeste in https://github.com/minio/minio/pull/15734
    • lifecycle: Assign unique id to rules with empty id by @krisis in https://github.com/minio/minio/pull/15731
    • Freeze before exit when _MINIO_DEBUG_BLOCK_ON_FATAL_CRASH is defined by @vadmeste in https://github.com/minio/minio/pull/15709

    New Contributors

    • @mersl made their first contribution in https://github.com/minio/minio/pull/15599
    • @kasuboski made their first contribution in https://github.com/minio/minio/pull/15698
    • @reivaj05 made their first contribution in https://github.com/minio/minio/pull/15673

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-09-17T00-09-45Z...RELEASE.2022-09-22T18-57-27Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-09-17T00-09-45Z(Sep 17, 2022)

    What's Changed

    • limit number of buckets to 500k by @harshavardhana in https://github.com/minio/minio/pull/15668
    • Add StopDiskAtLimit by @klauspost in https://github.com/minio/minio/pull/15670
    • add vulncheck action to CI by @aead in https://github.com/minio/minio/pull/15676
    • Persist in-memory replication stats to disk by @poornas in https://github.com/minio/minio/pull/15594
    • prometheus: Track errors during REST read/write calls by @vadmeste in https://github.com/minio/minio/pull/15678
    • Fix S3 Select CSV -> JSON with variable field count by @klauspost in https://github.com/minio/minio/pull/15677
    • use go1.19.1 for vulncheck by @harshavardhana in https://github.com/minio/minio/pull/15680
    • Pass groups claim into condition values by @donatello in https://github.com/minio/minio/pull/15679
    • panic if replication config could not be read from disk by @poornas in https://github.com/minio/minio/pull/15685
    • update docs to add cycle in scanner by @sinhaashish in https://github.com/minio/minio/pull/15686
    • Fix flaky TestGetObjectWithOutdatedDisks by @klauspost in https://github.com/minio/minio/pull/15687
    • When listing, do not count delete markers by @klauspost in https://github.com/minio/minio/pull/15689
    • change replication stats save path by @poornas in https://github.com/minio/minio/pull/15690
    • add pre-conditions support for PUT calls during replication by @harshavardhana in https://github.com/minio/minio/pull/15674
    • optimize double reads by reusing results from checkUploadIDExists() by @harshavardhana in https://github.com/minio/minio/pull/15692
    • upgrade all deps and update CREDITS by @harshavardhana in https://github.com/minio/minio/pull/15650

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-09-07T22-25-02Z...RELEASE.2022-09-17T00-09-45Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-09-07T22-25-02Z(Sep 8, 2022)

    What's Changed

    • avoid printing io.ErrUnexpectedEOF for .metacache objects by @harshavardhana in https://github.com/minio/minio/pull/15642
    • do not print object not found errors in MRF healing by @harshavardhana in https://github.com/minio/minio/pull/15646
    • fis deadlock caused by unlocking a non-existing lock by @yudoutingle in https://github.com/minio/minio/pull/15635
    • fix: use optimized json.NewEncoder instead for metrics by @harshavardhana in https://github.com/minio/minio/pull/15648
    • Reuse madmin.ClusterRegistrationInfo by @anjalshireesh in https://github.com/minio/minio/pull/15654
    • verify RenameData() versions to be consistent by @harshavardhana in https://github.com/minio/minio/pull/15649
    • allow non-standards fallback for Expires header by @harshavardhana in https://github.com/minio/minio/pull/15655
    • Helm Chart: Use a Prometheus Operator Probe to scrape Cluster metrics by @briend in https://github.com/minio/minio/pull/15659
    • allow non-standards fallback for all http.TimeFormats by @harshavardhana in https://github.com/minio/minio/pull/15662
    • fix: trigger Heal when xl.meta needs healing during PUT by @harshavardhana in https://github.com/minio/minio/pull/15661
    • serialize updates to healing tracker by @krisis in https://github.com/minio/minio/pull/15647
    • Update PR template to make it easy to request for doc changes by @vadmeste in https://github.com/minio/minio/pull/15665

    New Contributors

    • @yudoutingle made their first contribution in https://github.com/minio/minio/pull/15635
    • @briend made their first contribution in https://github.com/minio/minio/pull/15659

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-09-01T23-53-36Z...RELEASE.2022-09-07T22-25-02Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-09-01T23-53-36Z(Sep 2, 2022)

    What's Changed

    • helm: add extra volumes and extra volume mounts option for job resources by @cowjen01 in https://github.com/minio/minio/pull/15590
    • improve performance for inlined data by @harshavardhana in https://github.com/minio/minio/pull/15603
    • site replication: Avoid returning root svcacct info in sr metadata by @poornas in https://github.com/minio/minio/pull/15608
    • Add extended checksum by @klauspost in https://github.com/minio/minio/pull/15433
    • audit-log: Add time to get object from remote-tier by @krisis in https://github.com/minio/minio/pull/15602
    • Avoid messages from go test output by @vadmeste in https://github.com/minio/minio/pull/15601
    • chore: remove duplicate word in comments by @Abirdcfly in https://github.com/minio/minio/pull/15607
    • feat: Add healing trace by @klauspost in https://github.com/minio/minio/pull/15616
    • Encrypt checksums by @klauspost in https://github.com/minio/minio/pull/15620
    • fix: add healing metric type for heal tracing by @harshavardhana in https://github.com/minio/minio/pull/15631
    • audit: account for response headers separately by @krisis in https://github.com/minio/minio/pull/15610
    • tier: avoid stats infinite loop in forwardTo method by @harshavardhana in https://github.com/minio/minio/pull/15640
    • Avoid replicating dirs in listing with replication enabled by @vadmeste in https://github.com/minio/minio/pull/15641
    • Save resync status in the backend using a last update timestamp by @vadmeste in https://github.com/minio/minio/pull/15638

    New Contributors

    • @Abirdcfly made their first contribution in https://github.com/minio/minio/pull/15607

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-26T19-53-15Z...RELEASE.2022-09-01T23-53-36Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-26T19-53-15Z(Aug 29, 2022)

    What's Changed

    • fix: validate deployment id when adding peer clusters by @poornas in https://github.com/minio/minio/pull/15591
    • Add concurrency to healing objects on a fresh disk by @krisis in https://github.com/minio/minio/pull/15575
    • fix key 'linters-settings' is duplicated by @jiuker in https://github.com/minio/minio/pull/15589
    • upgrade golang-lint to the latest by @harshavardhana in https://github.com/minio/minio/pull/15600
    • move adding port location so a pool which consist only one host got the port information by @FwP-IDN in https://github.com/minio/minio/pull/15585

    New Contributors

    • @FwP-IDN made their first contribution in https://github.com/minio/minio/pull/15585

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-25T07-17-05Z...RELEASE.2022-08-26T19-53-15Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-25T07-17-05Z(Aug 25, 2022)

    What's Changed

    • Properly replicate policy mapping for virtual users by @vadmeste in https://github.com/minio/minio/pull/15558
    • Supply proper initial values for annotations by @dingxiangfei2009 in https://github.com/minio/minio/pull/15570
    • helm: add extra volumes and extra volume mounts option by @cowjen01 in https://github.com/minio/minio/pull/15568
    • use new xxml for XML responses to support rare control characters by @harshavardhana in https://github.com/minio/minio/pull/15511
    • Factor out external event notification funcs by @donatello in https://github.com/minio/minio/pull/15574
    • Add listing block reuse by @klauspost in https://github.com/minio/minio/pull/15579
    • Fix regression in notifying peers about new policy mapping by @vadmeste in https://github.com/minio/minio/pull/15583
    • Ensure that only one node performs site replication healing by @vadmeste in https://github.com/minio/minio/pull/15584
    • fix: objects matching prefixes should not leave delete markers by @harshavardhana in https://github.com/minio/minio/pull/15586
    • Allow querying a single target in config get API by @donatello in https://github.com/minio/minio/pull/15587

    New Contributors

    • @dingxiangfei2009 made their first contribution in https://github.com/minio/minio/pull/15570

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-22T23-53-06Z...RELEASE.2022-08-25T07-17-05Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-22T23-53-06Z(Aug 23, 2022)

    Highlights

    • Versioning now supports Idempotent delete markers #15521
    • Disk IO statistics is now available as real-time monitoring via mc #15512
    • Replication now more accurately tracks failed replication events #15556

    What's Changed

    • helm: fix correct deployment environment variable by @sebtiz13 in https://github.com/minio/minio/pull/15534
    • Introduce disk io stats metrics by @vadmeste in https://github.com/minio/minio/pull/15512
    • fix: trim arn:aws:kms from incoming SSE aws-kms-key-id by @harshavardhana in https://github.com/minio/minio/pull/15540
    • replication: centralize healthcheck for remote targets by @poornas in https://github.com/minio/minio/pull/15516
    • Ensure subnet+callhome configs are single sub-sys by @donatello in https://github.com/minio/minio/pull/15542
    • mount export volume even if persistence is disabled by @rootmout in https://github.com/minio/minio/pull/15543
    • fix: increase concurrency of DeleteObjects() to N/10th by @harshavardhana in https://github.com/minio/minio/pull/15546
    • avoid using errors.As for 'errors.New' use errors.Is by @harshavardhana in https://github.com/minio/minio/pull/15549
    • Fixes after moving config parsing code in madmin by @donatello in https://github.com/minio/minio/pull/15552
    • feat: add idempotent delete marker support by @harshavardhana in https://github.com/minio/minio/pull/15521
    • fix: walk() should cancel itself upon context cancellation by @harshavardhana in https://github.com/minio/minio/pull/15553
    • typo in parameter name by @agowa338 in https://github.com/minio/minio/pull/15555
    • change default lock retry interval to 50ms by @harshavardhana in https://github.com/minio/minio/pull/15560
    • Fix error types in internal config package by @donatello in https://github.com/minio/minio/pull/15559
    • fix: ensure metadata update happens after deletemarker replication by @poornas in https://github.com/minio/minio/pull/15564
    • replication: improve MRF healing. by @poornas in https://github.com/minio/minio/pull/15556

    New Contributors

    • @sebtiz13 made their first contribution in https://github.com/minio/minio/pull/15534
    • @rootmout made their first contribution in https://github.com/minio/minio/pull/15543
    • @agowa338 made their first contribution in https://github.com/minio/minio/pull/15555

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-13T21-54-44Z...RELEASE.2022-08-22T23-53-06Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-13T21-54-44Z(Aug 14, 2022)

    What's Changed

    • Fix env var output in config get/export APIs by @donatello in https://github.com/minio/minio/pull/15528
    • init console after IAM init() by @harshavardhana in https://github.com/minio/minio/pull/15531
    • Move some config constants to madmin by @donatello in https://github.com/minio/minio/pull/15529
    • helm: (Bugfix) enable objectlocking by @reyerdam in https://github.com/minio/minio/pull/15524

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-11T04-37-28Z...RELEASE.2022-08-13T21-54-44Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-11T04-37-28Z(Aug 12, 2022)

    What's Changed

    • fix: maxConcurrent '0' is an invalid value by @harshavardhana in https://github.com/minio/minio/pull/15500
    • LDAP/OpenID must be initialized IAM Init() by @harshavardhana in https://github.com/minio/minio/pull/15491
    • restrict number of disks used for scanning buckets upto GOMAXPROCS by @harshavardhana in https://github.com/minio/minio/pull/15492
    • replication: heal proactively upon access by @poornas in https://github.com/minio/minio/pull/15501
    • site replication: fix group updates to set status correctly. by @poornas in https://github.com/minio/minio/pull/15507
    • fix: incorrect context timeout during listPath() by @harshavardhana in https://github.com/minio/minio/pull/15509
    • Upgrade madmin-go to latest version (v1.4.15) by @anjalshireesh in https://github.com/minio/minio/pull/15510
    • helm: Support adding objectlocking for buckets by @reyerdam in https://github.com/minio/minio/pull/15505
    • chore: bad title syntax of helm chart README by @noahziheng in https://github.com/minio/minio/pull/15513
    • Add trace support for decommissioning by @krisis in https://github.com/minio/minio/pull/15502

    New Contributors

    • @reyerdam made their first contribution in https://github.com/minio/minio/pull/15505
    • @noahziheng made their first contribution in https://github.com/minio/minio/pull/15513

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-08T18-34-09Z...RELEASE.2022-08-11T04-37-28Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-08T18-34-09Z(Aug 8, 2022)

    What's Changed

    • remove IAM old migration code by @harshavardhana in https://github.com/minio/minio/pull/15476
    • helm fix: add a new line to the end of the credentials file when creating a user by @cowjen01 in https://github.com/minio/minio/pull/15485
    • fix: keep writing previous speedtest result by @harshavardhana in https://github.com/minio/minio/pull/15484
    • site replication: handle invalid endpoint errors by @poornas in https://github.com/minio/minio/pull/15499
    • remove non-working GetRawData() from FS mode by @harshavardhana in https://github.com/minio/minio/pull/15498

    New Contributors

    • @cowjen01 made their first contribution in https://github.com/minio/minio/pull/15485

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-05T23-27-09Z...RELEASE.2022-08-08T18-34-09Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-05T23-27-09Z(Aug 6, 2022)

    Highlights

    • mc admin config export now supports relevant configured environment variables, refer #15465
    • Object speedtest now supports latency measurements, refer #15479
    • Fixes a rare goroutine leak in IAM subsystem causing CPU spike, refer #15471

    What's Changed

    • Add support converting non-inlined to inlined by @harshavardhana in https://github.com/minio/minio/pull/15444
    • fix: log if there is readDir() failure with ListBuckets by @harshavardhana in https://github.com/minio/minio/pull/15461
    • Add OIDC to the HelmChart by @ktaf in https://github.com/minio/minio/pull/15469
    • Properly cancel listAllIAMConfigItems to avoid goroutine leak by @vadmeste in https://github.com/minio/minio/pull/15471
    • Replaces 'disk'=>'drive' visible to end user by @ebozduman in https://github.com/minio/minio/pull/15464
    • fix: decodeDirObject() in single drive DeleteObjects() call by @harshavardhana in https://github.com/minio/minio/pull/15477
    • Add env variables to configuration APIs output by @donatello in https://github.com/minio/minio/pull/15465
    • bucket metadata import: return all errors by @poornas in https://github.com/minio/minio/pull/15462
    • feat: add SpeedTest ResponseTimes and TTFB by @harshavardhana in https://github.com/minio/minio/pull/15479
    • Ensure lock/versioning permissions when creating a bucket by @vadmeste in https://github.com/minio/minio/pull/15432

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-02T23-59-16Z...RELEASE.2022-08-05T23-27-09Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-02T23-59-16Z(Aug 3, 2022)

    What's Changed

    • target: Set 20000 as maximum parallel target notif calls by @vadmeste in https://github.com/minio/minio/pull/15435
    • fix: resetting a config with a non default target name by @vadmeste in https://github.com/minio/minio/pull/15448
    • fix: allow P-384/P-512 constant time implementation by @harshavardhana in https://github.com/minio/minio/pull/15445
    • modify user secret volumes mount path name by @dormanze in https://github.com/minio/minio/pull/15443
    • fix: instrument os.OpenFile differently for Reads and Writes by @harshavardhana in https://github.com/minio/minio/pull/15449
    • fix: readdir fallback on root of the drive by @harshavardhana in https://github.com/minio/minio/pull/15457
    • do not crash for unwrapErrs return nil by @harshavardhana in https://github.com/minio/minio/pull/15456
    • bloom: healObject to mark a path dirty only for dangling objects by @vadmeste in https://github.com/minio/minio/pull/15458
    • kes: automatically reload KES client certificate by @aead in https://github.com/minio/minio/pull/15450

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-07-30T05-21-40Z...RELEASE.2022-08-02T23-59-16Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-07-30T05-21-40Z(Jul 30, 2022)

    What's Changed

    • allow root users to return appropriate policy in AccountInfo by @harshavardhana in https://github.com/minio/minio/pull/15437
    • fix: Incorrect ServersCount in cluster.info by @anjalshireesh in https://github.com/minio/minio/pull/15431

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-07-29T19-40-48Z...RELEASE.2022-07-30T05-21-40Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-07-29T19-40-48Z(Jul 29, 2022)

    Highlights

    • 'mc admin update' allowed reading path traversals on accessible files and folders, refer to our advisory
    • Go-routine leak fixed with kafka client library for audit and bucket events, refer #15434
    • Improvements in logging, refer #15408

    What's Changed

    • Fix typo in json key for DeploymentID by @anjalshireesh in https://github.com/minio/minio/pull/15406
    • Remove internal usage of consoleAdmin by @donatello in https://github.com/minio/minio/pull/15402
    • use logger.LogOnce to reduce printing disconnection logs by @harshavardhana in https://github.com/minio/minio/pull/15408
    • optimize speedtest for smaller setups by @harshavardhana in https://github.com/minio/minio/pull/15414
    • fix: cluster healthcheck for single drive setups by @harshavardhana in https://github.com/minio/minio/pull/15415
    • Optionally run a command when completing jobs by @andrewhibbert in https://github.com/minio/minio/pull/15017
    • chore(deps): Included dependency review by @naveensrinivasan in https://github.com/minio/minio/pull/14958
    • Do not update directory access time by @vadmeste in https://github.com/minio/minio/pull/15426
    • replication: Include Replica object versions in resync by @poornas in https://github.com/minio/minio/pull/15427
    • do not allow filesystem fallback in server download by @harshavardhana in https://github.com/minio/minio/pull/15429
    • Fix passing application configuration to console by @donatello in https://github.com/minio/minio/pull/15409
    • choose starting concurrency based on number of local disks by @harshavardhana in https://github.com/minio/minio/pull/15428
    • Download binary once and push it to other servers by @cniackz in https://github.com/minio/minio/pull/15407
    • fix: regression in resolving the right versions by @harshavardhana in https://github.com/minio/minio/pull/15430
    • Update kafka library by @vadmeste in https://github.com/minio/minio/pull/15434

    New Contributors

    • @andrewhibbert made their first contribution in https://github.com/minio/minio/pull/15017

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-07-26T00-53-03Z...RELEASE.2022-07-29T19-40-48Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-07-26T00-53-03Z(Jul 26, 2022)

    What's Changed

    • Add cluster info to inspect/profiling archive by @vadmeste in https://github.com/minio/minio/pull/15360
    • test: use T.TempDir to create temporary test directory by @Juneezee in https://github.com/minio/minio/pull/15400
    • move all CI/CD to go1.18 by @harshavardhana in https://github.com/minio/minio/pull/15401
    • upgrade: Split in two steps to ensure a stable retry by @vadmeste in https://github.com/minio/minio/pull/15396
    • site replication: fix healing of bucket deletes. by @poornas in https://github.com/minio/minio/pull/15377
    • fix:wrapped_error_will_not_ignore by @jiuker in https://github.com/minio/minio/pull/15387

    New Contributors

    • @Juneezee made their first contribution in https://github.com/minio/minio/pull/15400

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-07-24T17-09-31Z...RELEASE.2022-07-26T00-53-03Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-07-24T17-09-31Z(Jul 24, 2022)

    What's Changed

    • fix: fork os.MkdirAll to optimize cases where parent exists by @harshavardhana in https://github.com/minio/minio/pull/15379
    • Add compression scheme to header by @klauspost in https://github.com/minio/minio/pull/15395

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-07-24T01-54-52Z...RELEASE.2022-07-24T17-09-31Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-07-24T01-54-52Z(Jul 24, 2022)

    Highlights

    • Major performance improvements for single large objects up to 5TiB with concurrent uploads refer #15346, #15353, #15375

    What's Changed

    • Console v0.19.1 by @dvaldivia in https://github.com/minio/minio/pull/15338
    • Fix issue when a Helm create user job returns more than once by @jhaynie in https://github.com/minio/minio/pull/15321
    • export IAM: include site replicator svcacct by @poornas in https://github.com/minio/minio/pull/15339
    • kms: add context.Context to KMS API calls by @aead in https://github.com/minio/minio/pull/15327
    • Independent Multipart Uploads by @klauspost in https://github.com/minio/minio/pull/15346
    • Fix reused field crash by @klauspost in https://github.com/minio/minio/pull/15351
    • use partInfo only for intermediate part.x.meta by @harshavardhana in https://github.com/minio/minio/pull/15353
    • remove unnecessary logs in IAM store by @harshavardhana in https://github.com/minio/minio/pull/15356
    • fix: allow payload verification error to be returned by @harshavardhana in https://github.com/minio/minio/pull/15364
    • fix: readQuorum calculation when defaultParityCount is 0 by @harshavardhana in https://github.com/minio/minio/pull/15363
    • Fix multiple validateAdminReq actions by @klauspost in https://github.com/minio/minio/pull/15372
    • replication: Add API to return unreplicated objects by @poornas in https://github.com/minio/minio/pull/15091
    • tests: Add context cancelation by @klauspost in https://github.com/minio/minio/pull/15374
    • update grafana dashboard by @vashian in https://github.com/minio/minio/pull/15357
    • Complete multipart uploads faster on encrypted backends by @klauspost in https://github.com/minio/minio/pull/15375
    • upgrade mqtt library to v1.4.1 by @harshavardhana in https://github.com/minio/minio/pull/15366
    • save IAM export assets at a folder prefix by @poornas in https://github.com/minio/minio/pull/15355
    • fix: Calculated expression error by @jiuker in https://github.com/minio/minio/pull/15362
    • single drive: Remove bucket metadata on DeleteBucket by @poornas in https://github.com/minio/minio/pull/15378
    • Allow site replication config with multiple IDPs by @donatello in https://github.com/minio/minio/pull/15361
    • helm: fix regression of #15243 by @sathieu in https://github.com/minio/minio/pull/15386
    • Add runtimeClassName support to vanilla helm chart by @dnskr in https://github.com/minio/minio/pull/15385
    • support additional claim info in Auditing STS calls by @harshavardhana in https://github.com/minio/minio/pull/15381
    • Allow DelKVS to delete specific sub-system fields. by @taran-p in https://github.com/minio/minio/pull/15354
    • Pass multiple IDP config to console by @donatello in https://github.com/minio/minio/pull/15270
    • Console v0.19.2 by @dvaldivia in https://github.com/minio/minio/pull/15390

    New Contributors

    • @jhaynie made their first contribution in https://github.com/minio/minio/pull/15321
    • @vashian made their first contribution in https://github.com/minio/minio/pull/15357
    • @jiuker made their first contribution in https://github.com/minio/minio/pull/15362
    • @dnskr made their first contribution in https://github.com/minio/minio/pull/15385
    • @taran-p made their first contribution in https://github.com/minio/minio/pull/15354

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-07-17T15-43-14Z...RELEASE.2022-07-24T01-54-52Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-07-17T15-43-14Z(Jul 17, 2022)

    What's Changed

    • add reconnect duration allows for verifying disconnect intervals by @harshavardhana in https://github.com/minio/minio/pull/15306
    • copy query values before update to preserve the expected keys by @harshavardhana in https://github.com/minio/minio/pull/15310
    • support 'admin update' for hotfix versions by @harshavardhana in https://github.com/minio/minio/pull/15308
    • allow decommission to continue when healing by @harshavardhana in https://github.com/minio/minio/pull/15312
    • Allow topologySpreadConstraints configuration by @justusbunsi in https://github.com/minio/minio/pull/14684
    • Fix failure in createUser used by make-user-job helm chart by @lamida in https://github.com/minio/minio/pull/15293
    • add annotations to ServiceMonitor by @sathieu in https://github.com/minio/minio/pull/15020
    • Fix data race, which caused tests execution to fail by @shichanglin5 in https://github.com/minio/minio/pull/15313
    • Exclude upload-ids with incomplete part upload in multipart listing by @vadmeste in https://github.com/minio/minio/pull/15318
    • support encrypted/compressed objects properly during decommission by @harshavardhana in https://github.com/minio/minio/pull/15320
    • Add decommission compression+encryption enabled tests by @harshavardhana in https://github.com/minio/minio/pull/15322

    New Contributors

    • @justusbunsi made their first contribution in https://github.com/minio/minio/pull/14684
    • @lamida made their first contribution in https://github.com/minio/minio/pull/15293
    • @sathieu made their first contribution in https://github.com/minio/minio/pull/15020

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-07-15T03-44-22Z...RELEASE.2022-07-17T15-43-14Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-07-15T03-44-22Z(Jul 15, 2022)

    What's Changed

    • correct RefreshCall & UnlockCall of DefaultTimeouts by @shichanglin5 in https://github.com/minio/minio/pull/15288
    • listing: Expire object versions past expiry by @krisis in https://github.com/minio/minio/pull/15287
    • Updating minio-go by @cniackz in https://github.com/minio/minio/pull/15297
    • Fix site replication healing of missing buckets by @poornas in https://github.com/minio/minio/pull/15298
    • enable using different ports for minioAPIPort/service.port and minioConsolePort/consoleService.port by @chel-ou in https://github.com/minio/minio/pull/15259
    • Add missing TLS config to service monitor by @OvervCW in https://github.com/minio/minio/pull/15228
    • Default DeleteReplication rule status if unspecified. by @poornas in https://github.com/minio/minio/pull/15301
    • fix: skip objects expired via lifecycle rules during decommission by @harshavardhana in https://github.com/minio/minio/pull/15300
    • allow force delete on decom pool by @harshavardhana in https://github.com/minio/minio/pull/15302

    New Contributors

    • @shichanglin5 made their first contribution in https://github.com/minio/minio/pull/15288
    • @cniackz made their first contribution in https://github.com/minio/minio/pull/15297
    • @chel-ou made their first contribution in https://github.com/minio/minio/pull/15259
    • @OvervCW made their first contribution in https://github.com/minio/minio/pull/15228

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-07-13T23-29-44Z...RELEASE.2022-07-15T03-44-22Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-07-13T23-29-44Z(Jul 14, 2022)

    What's Changed

    • Adding error check for jetstream connection by @rayjanoka in https://github.com/minio/minio/pull/15252
    • Fix Helm chart does not support special characters in access/secret key by @dormanze in https://github.com/minio/minio/pull/15243
    • do not recreate deprecated delete-journal.bin, only read it by @harshavardhana in https://github.com/minio/minio/pull/15185
    • remove auto-port warning for console-address by @harshavardhana in https://github.com/minio/minio/pull/15260
    • Allow compaction at bucket top level. by @klauspost in https://github.com/minio/minio/pull/15266
    • fix grammatic errors and minor rewrites by @daniel-bogusz95 in https://github.com/minio/minio/pull/15264
    • Purge the objects immediately when x-minio-force-delete header is passed in DeleteObject and DeleteBucket API by @Praveenrajmani in https://github.com/minio/minio/pull/15148
    • metric: add KMS-related metrics by @aead in https://github.com/minio/minio/pull/15258
    • site replication: Ignore missing targets/replication config during re… by @poornas in https://github.com/minio/minio/pull/15269
    • Add compressed file index by @klauspost in https://github.com/minio/minio/pull/15247
    • Add 4K minimum compressed size by @klauspost in https://github.com/minio/minio/pull/15273
    • Do not print erasure read log when a disk becomes offline by @vadmeste in https://github.com/minio/minio/pull/15277
    • allow custom speedtest bucket by @harshavardhana in https://github.com/minio/minio/pull/15271
    • fix: simplify passing auditLog eventType by @harshavardhana in https://github.com/minio/minio/pull/15278
    • Avoid listing buckets from a suspended pool by @vadmeste in https://github.com/minio/minio/pull/15283
    • preserve incoming query params in success_action_redirect by @harshavardhana in https://github.com/minio/minio/pull/15280
    • Upgrade compression package by @klauspost in https://github.com/minio/minio/pull/15284
    • Add padding to compressed+encrypted files by @klauspost in https://github.com/minio/minio/pull/15282
    • resync: Avoid concurrent access/write on map by @poornas in https://github.com/minio/minio/pull/15286
    • Add a github workflow test for root disk detection by @vadmeste in https://github.com/minio/minio/pull/15267

    New Contributors

    • @rayjanoka made their first contribution in https://github.com/minio/minio/pull/15252
    • @daniel-bogusz95 made their first contribution in https://github.com/minio/minio/pull/15264

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-07-08T00-05-23Z...RELEASE.2022-07-13T23-29-44Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-07-08T00-05-23Z(Jul 8, 2022)

    What's Changed

    • retry and resume decom operation upon retriable failures by @harshavardhana in https://github.com/minio/minio/pull/15244
    • remove unnecessary nAgreed value in partial() func by @harshavardhana in https://github.com/minio/minio/pull/15242
    • fix: do not pass around sync.Map by @harshavardhana in https://github.com/minio/minio/pull/15250
    • fix: rootdisk detection by not using cached value when GetDiskInfo() errors out by @vadmeste in https://github.com/minio/minio/pull/15249

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-07-06T20-29-49Z...RELEASE.2022-07-08T00-05-23Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-07-06T20-29-49Z(Jul 7, 2022)

    Highlights

    • Major scanner metrics overhaul now you can watch scanner metrics in realtime using mc admin top scanner #15161
    • Multiple external IDP management simplification #15152
    • Decommission now supports mutations such as delete, update of metadata on objects from an active decom pool.

    What's Changed

    • fix: decommission delete markers for non-current objects by @harshavardhana in https://github.com/minio/minio/pull/15225
    • service port set to minioAPIPort in helm by @dormanze in https://github.com/minio/minio/pull/15223
    • Cache disk metrics by @klauspost in https://github.com/minio/minio/pull/15229
    • Fix missing whitespace in error message for IncompleteBody by @haslersn in https://github.com/minio/minio/pull/15227
    • Add detailed scanner metrics by @klauspost in https://github.com/minio/minio/pull/15161
    • Add external IDP management Admin API for OpenID by @donatello in https://github.com/minio/minio/pull/15152
    • fix: calculate scanner metric paths for single drive by @harshavardhana in https://github.com/minio/minio/pull/15232
    • fix: allow certain mutations on objects during decommissioning by @harshavardhana in https://github.com/minio/minio/pull/15231
    • no 'replicate status' without replication config by @harshavardhana in https://github.com/minio/minio/pull/15233
    • add NATS JetStream support by @harshavardhana in https://github.com/minio/minio/pull/15201
    • Better reporting of total/free usable capacity of the cluster by @vadmeste in https://github.com/minio/minio/pull/15230

    New Contributors

    • @dormanze made their first contribution in https://github.com/minio/minio/pull/15223
    • @haslersn made their first contribution in https://github.com/minio/minio/pull/15227

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-07-04T21-02-54Z...RELEASE.2022-07-06T20-29-49Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-07-04T21-02-54Z(Jul 4, 2022)

    What's Changed

    • Simplify Prometheus metrics gather by @harshavardhana in https://github.com/minio/minio/pull/15210
    • site healing: Skip stale iam asset updates from peer. by @poornas in https://github.com/minio/minio/pull/15203
    • return appropriate error under 'decom status' by @harshavardhana in https://github.com/minio/minio/pull/15213
    • do not print errFileNotFound in entries.resolve() by @harshavardhana in https://github.com/minio/minio/pull/15216
    • decom IAM, Bucket metadata properly by @harshavardhana in https://github.com/minio/minio/pull/15220

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-06-30T20-58-09Z...RELEASE.2022-07-04T21-02-54Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-06-30T20-58-09Z(Jul 1, 2022)

    Highlights

    • Supports parity configuration of '0', '1' enabling support for 2,3 drive setups for erasure coding.

    What's Changed

    • add license banner for GNU AGPLv3 by @harshavardhana in https://github.com/minio/minio/pull/15178
    • save MinIO version with each version (8-bytes extra) by @harshavardhana in https://github.com/minio/minio/pull/15170
    • Remove unnecessary code in WalkDir() by @vadmeste in https://github.com/minio/minio/pull/15168
    • perform object sweep after equeue the latest CopyObject() by @harshavardhana in https://github.com/minio/minio/pull/15183
    • Upgrade reedsolomon/compression packages by @klauspost in https://github.com/minio/minio/pull/15182
    • Revert proxying requests with precondition errors by @vadmeste in https://github.com/minio/minio/pull/15180
    • allow support for parity '0', '1' enabling support for 2,3 drive setups by @harshavardhana in https://github.com/minio/minio/pull/15171
    • handle missing funcNames for handlers by @harshavardhana in https://github.com/minio/minio/pull/15188
    • update banner to reflect the final agreed UI by @harshavardhana in https://github.com/minio/minio/pull/15192
    • site healing: Skip stale bucket metadata updates from peer by @poornas in https://github.com/minio/minio/pull/15186
    • de-couple caller context to avoid internal races by @harshavardhana in https://github.com/minio/minio/pull/15195
    • Save minio version in xl.meta while healing in outdated disks by @vadmeste in https://github.com/minio/minio/pull/15181
    • finalize startup-banner and remove unnecessary logs by @harshavardhana in https://github.com/minio/minio/pull/15202
    • Make ReqInfo concurrency safe by @klauspost in https://github.com/minio/minio/pull/15204
    • update banner with version+runtime by @harshavardhana in https://github.com/minio/minio/pull/15206

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-06-25T15-50-16Z...RELEASE.2022-06-30T20-58-09Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-06-25T15-50-16Z(Jun 26, 2022)

    What's Changed

    • remove gateway banner and some other TODO loggers by @harshavardhana in https://github.com/minio/minio/pull/15125
    • s3-verify: Fix endpoint and missing comparison by @vadmeste in https://github.com/minio/minio/pull/15129
    • Parallelize new disks healing of different erasure sets by @vadmeste in https://github.com/minio/minio/pull/15112
    • fips: simplify TLS configuration by @aead in https://github.com/minio/minio/pull/15127
    • fix: cant parse comment without '=' in environment file by @anoworl in https://github.com/minio/minio/pull/15130
    • add go1.18 specific curve preferences by @harshavardhana in https://github.com/minio/minio/pull/15132
    • s3/zip: extract metadata properly for Zipped objects by @harshavardhana in https://github.com/minio/minio/pull/15123
    • replication: Avoid proxying when precondition failed by @poornas in https://github.com/minio/minio/pull/15134
    • CopyObject: fix read/write err on closed pipe by @poornas in https://github.com/minio/minio/pull/15135
    • fips: fix order of elliptic curves by @aead in https://github.com/minio/minio/pull/15141
    • s3-verify: Add a flag to exclude younger than a certain age by @vadmeste in https://github.com/minio/minio/pull/15142
    • use signature-v2 for 'object perf' tests to avoid CPU using sha256 by @harshavardhana in https://github.com/minio/minio/pull/15151
    • heal: Include dir markers when healing a fresh disk by @vadmeste in https://github.com/minio/minio/pull/15158
    • Add API's to import/export IAM data by @poornas in https://github.com/minio/minio/pull/15014
    • reduce sha256 CPU usage by turning it off for speedtests by @harshavardhana in https://github.com/minio/minio/pull/15154
    • Trace all http requests by @vadmeste in https://github.com/minio/minio/pull/15064
    • Fix health client leak when calling UpdateAllTargets by @klauspost in https://github.com/minio/minio/pull/15167
    • fix: change timedValue to return the previously cached value by @harshavardhana in https://github.com/minio/minio/pull/15169

    New Contributors

    • @anoworl made their first contribution in https://github.com/minio/minio/pull/15130

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-06-20T23-13-45Z...RELEASE.2022-06-25T15-50-16Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-06-20T23-13-45Z(Jun 21, 2022)

    What's Changed

    • Update gopsutil to v3.22.5 by @danieljakots in https://github.com/minio/minio/pull/15098
    • fix: ignore printing io.EOF during WalkDir() on concurrently modified objects by @harshavardhana in https://github.com/minio/minio/pull/15100
    • Avoid CompleteMultipart freeze with unexpected network issue by @vadmeste in https://github.com/minio/minio/pull/15102
    • Add generic function to retrieve config value with metadata by @donatello in https://github.com/minio/minio/pull/15083
    • fix typo in site replication version healing by @poornas in https://github.com/minio/minio/pull/15103
    • Add import/export APIs to migrate bucket metadata by @poornas in https://github.com/minio/minio/pull/14929
    • Update Console to v0.19.0 by @dvaldivia in https://github.com/minio/minio/pull/15109
    • heal: Report correctly in multip-pools setup by @vadmeste in https://github.com/minio/minio/pull/15117
    • debug/s3-verify: simplify the tool to use lower memory footprint by @harshavardhana in https://github.com/minio/minio/pull/15110
    • cleanup site replication error handling by @harshavardhana in https://github.com/minio/minio/pull/15113
    • fix: add 'mc support inspect' support for single drive deployment by @harshavardhana in https://github.com/minio/minio/pull/15122
    • improve error message for bucket metadata export/import API by @poornas in https://github.com/minio/minio/pull/15120

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-06-17T02-00-35Z...RELEASE.2022-06-20T23-13-45Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-06-17T02-00-35Z(Jun 17, 2022)

    What's Changed

    • debug: Close object after check by @vadmeste in https://github.com/minio/minio/pull/15077
    • kms: initialize after cli parsing by @vadmeste in https://github.com/minio/minio/pull/15076
    • Support configuration of a session policy file by @rgcostea in https://github.com/minio/minio/pull/15078
    • erasure-sd: Evaluate versioning Prefix in multi-delete objects by @vadmeste in https://github.com/minio/minio/pull/15081
    • fix: simplify healthcheck code to freeze calls only once by @harshavardhana in https://github.com/minio/minio/pull/15082
    • add bucket level S3 received/sent bytes by @harshavardhana in https://github.com/minio/minio/pull/15084
    • fix: background local test also via channel by @harshavardhana in https://github.com/minio/minio/pull/15086
    • add dataErrs for healing debug log by @harshavardhana in https://github.com/minio/minio/pull/15092
    • update '-v' flag behavior to include copyRight and license by @harshavardhana in https://github.com/minio/minio/pull/15097
    • Init site replication subsys after loading metadata by @poornas in https://github.com/minio/minio/pull/15099

    New Contributors

    • @rgcostea made their first contribution in https://github.com/minio/minio/pull/15078

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-06-11T19-55-32Z...RELEASE.2022-06-17T02-00-35Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-06-11T19-55-32Z(Jun 11, 2022)

    What's Changed

    • debug: verify diff on latest objects on source and target buckets by @harshavardhana in https://github.com/minio/minio/pull/15069
    • fix: add missing minio_s3_requests_total by @harshavardhana in https://github.com/minio/minio/pull/15070
    • Update dperf to 0.4.1 by @vadmeste in https://github.com/minio/minio/pull/15071
    • sts: Avoid printing all STS errors by @vadmeste in https://github.com/minio/minio/pull/15065

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-06-10T16-59-15Z...RELEASE.2022-06-11T19-55-32Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-06-10T16-59-15Z(Jun 10, 2022)

    What's Changed

    • Include the entirety of vendor folder in .gitignore by @kaankabalak in https://github.com/minio/minio/pull/15046
    • relax O_DIRECT in single drive mode if unsupported by @harshavardhana in https://github.com/minio/minio/pull/15045
    • Bump Console v0.18.1 by @dvaldivia in https://github.com/minio/minio/pull/15051
    • healthcheck cluster endpoint should honor write/readQuorum per pool by @harshavardhana in https://github.com/minio/minio/pull/15053
    • fix: make metrics endpoint responsive by reducing the chatter by @harshavardhana in https://github.com/minio/minio/pull/15055
    • prometheus: Add S3 4xx and 5xx S3 monitoring by @vadmeste in https://github.com/minio/minio/pull/15052
    • fix: for frozen calls return if client disconnects by @harshavardhana in https://github.com/minio/minio/pull/15062
    • support reading systemctl config automatically on baremetal setups by @harshavardhana in https://github.com/minio/minio/pull/15066

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-06-07T00-33-41Z...RELEASE.2022-06-10T16-59-15Z

    Source code(tar.gz)
    Source code(zip)
Owner
High Performance, Kubernetes Native Object Storage
High Performance, Kubernetes Native Object Storage
GeeseFS is a high-performance, POSIX-ish S3 (Yandex, Amazon) file system written in Go

GeeseFS is a high-performance, POSIX-ish S3 (Yandex, Amazon) file system written in Go Overview GeeseFS allows you to mount an S3 bucket as a file sys

Yandex.Cloud 292 Sep 27, 2022
Go object files

Go object files C developers will be familiar with compiling C source files (.c) into object files (.o) before linking them into their final form. Did

Andrew Kutz 0 Feb 1, 2022
Abstract File Storage

afs - abstract file storage Please refer to CHANGELOG.md if you encounter breaking changes. Motivation Introduction Usage Matchers Content modifiers S

Viant, Inc 206 Sep 27, 2022
A PDF document generator with high level support for text, drawing and images

GoFPDF document generator Package go-pdf/fpdf implements a PDF document generator with high level support for text, drawing and images. Features UTF-8

null 155 Sep 7, 2022
High Performance, Kubernetes Native Object Storage

MinIO Quickstart Guide MinIO is a High Performance Object Storage released under Apache License v2.0. It is API compatible with Amazon S3 cloud storag

High Performance, Kubernetes Native Object Storage 35.5k Sep 26, 2022
High Performance, Kubernetes Native Object Storage

MinIO Quickstart Guide MinIO is a High Performance Object Storage released under GNU Affero General Public License v3.0. It is API compatible with Ama

Multi-Cloud Object Storage 35.6k Oct 2, 2022
A High Performance Object Storage released under Apache License

MinIO Quickstart Guide MinIO is a High Performance Object Storage released under Apache License v2.0. It is API compatible with Amazon S3 cloud storag

null 1 Sep 30, 2021
go-fastdfs 是一个简单的分布式文件系统(私有云存储),具有无中心、高性能,高可靠,免维护等优点,支持断点续传,分块上传,小文件合并,自动同步,自动修复。Go-fastdfs is a simple distributed file system (private cloud storage), with no center, high performance, high reliability, maintenance free and other advantages, support breakpoint continuation, block upload, small file merge, automatic synchronization, automatic repair.(similar fastdfs).

中文 English 愿景:为用户提供最简单、可靠、高效的分布式文件系统。 go-fastdfs是一个基于http协议的分布式文件系统,它基于大道至简的设计理念,一切从简设计,使得它的运维及扩展变得更加简单,它具有高性能、高可靠、无中心、免维护等优点。 大家担心的是这么简单的文件系统,靠不靠谱,可不

小张 3.2k Sep 22, 2022
🔑A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout (LSM+WAL) similar to Riak.

bitcask A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout

James Mills 10 Sep 26, 2022
the pluto is a gateway new time, high performance, high stable, high availability, easy to use

pluto the pluto is a gateway new time, high performance, high stable, high availability, easy to use Acknowledgments thanks nbio for providing low lev

mobus 2 Sep 19, 2021
QingStor Object Storage service support for go-storage

go-services-qingstor QingStor Object Storage service support for go-storage. Install go get github.com/minhjh/go-service-qingstor/v3 Usage import ( "

minhjh 0 Dec 13, 2021
topolvm operator provide kubernetes local storage which is light weight and high performance

Topolvm-Operator Topolvm-Operator is an open source cloud-native local storage orchestrator for Kubernetes, which bases on topolvm. Supported environm

Alauda.io 21 Jun 29, 2022
An high performance and ops-free local storage solution for Kubernetes.

Carina carina 是一个CSI插件,在Kubernetes集群中提供本地存储持久卷 项目状态:开发测试中 CSI Version: 1.3.0 Carina architecture 支持的环境 Kubernetes:1.20 1.19 1.18 Node OS:Linux Filesys

BoCloud 10 May 18, 2022
Carina: an high performance and ops-free local storage for kubernetes

Carina English | 中文 Background Storage systems are complex! There are more and more kubernetes native storage systems nowadays and stateful applicatio

null 485 Sep 27, 2022
High Performance Remote Object Service Engine

=============== Hprose is a High Performance Remote Object Service Engine. It is a modern, lightweight, cross-language, cross-platform, object-oriente

小马哥 615 Jul 27, 2022
The Container Storage Interface (CSI) Driver for Fortress Block Storage This driver allows you to use Fortress Block Storage with your container orchestrator

fortress-csi The Container Storage Interface (CSI) Driver for Fortress Block Storage This driver allows you to use Fortress Block Storage with your co

Fortress 0 Jan 23, 2022
High-Performance server for NATS, the cloud native messaging system.

NATS is a simple, secure and performant communications system for digital systems, services and devices. NATS is part of the Cloud Native Computing Fo

NATS - The Cloud Native Messaging System 11.5k Sep 24, 2022
High-Performance server for NATS, the cloud native messaging system.

NATS is a simple, secure and performant communications system for digital systems, services and devices. NATS is part of the Cloud Native Computing Fo

NATS - The Cloud Native Messaging System 11.6k Oct 1, 2022
Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package.

Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package. The library allows you to call Go service methods from PHP with a minimal footprint, structures and []byte support.

Spiral Scout 1.1k Sep 22, 2022