High Performance, Kubernetes Native Object Storage

Overview

MinIO Quickstart Guide

Slack Docker Pulls

MinIO

MinIO is a High Performance Object Storage released under Apache License v2.0. It is API compatible with Amazon S3 cloud storage service. Use MinIO to build high performance infrastructure for machine learning, analytics and application data workloads.

This README provides quickstart instructions on running MinIO on baremetal hardware, including Docker-based installations. For Kubernetes environments, use the MinIO Kubernetes Operator.

Docker Installation

Use the following commands to run a standalone MinIO server on a Docker container.

Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a minimum of 4 drives per MinIO server. See MinIO Erasure Code Quickstart Guide for more complete documentation.

Stable

Run the following command to run the latest stable image of MinIO on a Docker container using an ephemeral data volume:

docker run -p 9000:9000 minio/minio server /data

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

NOTE: To deploy MinIO on Docker with persistent storage, you must map local persistent directories from the host OS to the container using the docker -v option. For example, -v /mnt/data:/data maps the host OS drive at /mnt/data to /data on the Docker container.

Edge

Run the following command to run the bleeding-edge image of MinIO on a Docker container using an ephemeral data volume:

docker run -p 9000:9000 minio/minio:edge server /data

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

NOTE: To deploy MinIO on Docker with persistent storage, you must map local persistent directories from the host OS to the container using the docker -v option. For example, -v /mnt/data:/data maps the host OS drive at /mnt/data to /data on the Docker container.

macOS

Use the following commands to run a standalone MinIO server on macOS.

Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a minimum of 4 drives per MinIO server. See MinIO Erasure Code Quickstart Guide for more complete documentation.

Homebrew (recommended)

Run the following command to install the latest stable MinIO package using Homebrew. Replace /data with the path to the drive or directory in which you want MinIO to store data.

brew install minio/stable/minio
minio server /data

NOTE: If you previously installed minio using brew install minio then it is recommended that you reinstall minio from minio/stable/minio official repo instead.

brew uninstall minio
brew install minio/stable/minio

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

Binary Download

Use the following command to download and run a standalone MinIO server on macOS. Replace /data with the path to the drive or directory in which you want MinIO to store data.

wget https://dl.min.io/server/minio/release/darwin-amd64/minio
chmod +x minio
./minio server /data

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

GNU/Linux

Use the following command to run a standalone MinIO server on Linux hosts running 64-bit Intel/AMD architectures. Replace /data with the path to the drive or directory in which you want MinIO to store data.

wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
./minio server /data

Replace /data with the path to the drive or directory in which you want MinIO to store data.

The following table lists supported architectures. Replace the wget URL with the architecture for your Linux host.

Architecture URL
64-bit Intel/AMD https://dl.min.io/server/minio/release/linux-amd64/minio
64-bit ARM https://dl.min.io/server/minio/release/linux-arm64/minio
64-bit PowerPC LE (ppc64le) https://dl.min.io/server/minio/release/linux-ppc64le/minio
IBM Z-Series (S390X) https://dl.min.io/server/minio/release/linux-s390x/minio

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

NOTE: Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a minimum of 4 drives per MinIO server. See MinIO Erasure Code Quickstart Guide for more complete documentation.

Microsoft Windows

To run MinIO on 64-bit Windows hosts, download the MinIO executable from the following URL:

https://dl.min.io/server/minio/release/windows-amd64/minio.exe

Use the following command to run a standalone MinIO server on the Windows host. Replace D:\ with the path to the drive or directory in which you want MinIO to store data. You must change the terminal or powershell directory to the location of the minio.exe executable, or add the path to that directory to the system $PATH:

minio.exe server D:\

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

NOTE: Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a minimum of 4 drives per MinIO server. See MinIO Erasure Code Quickstart Guide for more complete documentation.

FreeBSD

MinIO does not provide an official FreeBSD binary. However, FreeBSD maintains an upstream release using pkg:

pkg install minio
sysrc minio_enable=yes
sysrc minio_disks=/home/user/Photos
service minio start

Install from Source

Use the following commands to compile and run a standalone MinIO server from source. Source installation is only intended for developers and advanced users. If you do not have a working Golang environment, please follow How to install Golang. Minimum version required is go1.16

GO111MODULE=on go get github.com/minio/minio

The MinIO deployment starts using default root credentials minioadmin:minioadmin. You can test the deployment using the MinIO Browser, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to http://127.0.0.1:9000 and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

You can also connect using any S3-compatible tool, such as the MinIO Client mc commandline tool. See Test using MinIO Client mc for more information on using the mc commandline tool. For application developers, see https://docs.min.io/docs/ and click MINIO SDKS in the navigation to view MinIO SDKs for supported languages.

NOTE: Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a minimum of 4 drives per MinIO server. See MinIO Erasure Code Quickstart Guide for more complete documentation.

MinIO strongly recommends against using compiled-from-source MinIO servers for production environments.

Deployment Recommendations

Allow port access for Firewalls

By default MinIO uses the port 9000 to listen for incoming connections. If your platform blocks the port by default, you may need to enable access to the port.

ufw

For hosts with ufw enabled (Debian based distros), you can use ufw command to allow traffic to specific ports. Use below command to allow access to port 9000

ufw allow 9000

Below command enables all incoming traffic to ports ranging from 9000 to 9010.

ufw allow 9000:9010/tcp

firewall-cmd

For hosts with firewall-cmd enabled (CentOS), you can use firewall-cmd command to allow traffic to specific ports. Use below commands to allow access to port 9000

firewall-cmd --get-active-zones

This command gets the active zone(s). Now, apply port rules to the relevant zones returned above. For example if the zone is public, use

firewall-cmd --zone=public --add-port=9000/tcp --permanent

Note that permanent makes sure the rules are persistent across firewall start, restart or reload. Finally reload the firewall for changes to take effect.

firewall-cmd --reload

iptables

For hosts with iptables enabled (RHEL, CentOS, etc), you can use iptables command to enable all traffic coming to specific ports. Use below command to allow access to port 9000

iptables -A INPUT -p tcp --dport 9000 -j ACCEPT
service iptables restart

Below command enables all incoming traffic to ports ranging from 9000 to 9010.

iptables -A INPUT -p tcp --dport 9000:9010 -j ACCEPT
service iptables restart

Pre-existing data

When deployed on a single drive, MinIO server lets clients access any pre-existing data in the data directory. For example, if MinIO is started with the command minio server /mnt/data, any pre-existing data in the /mnt/data directory would be accessible to the clients.

The above statement is also valid for all gateway backends.

Test MinIO Connectivity

Test using MinIO Browser

MinIO Server comes with an embedded web based object browser. Point your web browser to http://127.0.0.1:9000 to ensure your server has started successfully.

Screenshot

Test using MinIO Client mc

mc provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff etc. It supports filesystems and Amazon S3 compatible cloud storage services. Follow the MinIO Client Quickstart Guide for further instructions.

Upgrading MinIO

MinIO server supports rolling upgrades, i.e. you can update one MinIO instance at a time in a distributed cluster. This allows upgrades with no downtime. Upgrades can be done manually by replacing the binary with the latest release and restarting all servers in a rolling fashion. However, we recommend all our users to use mc admin update from the client. This will update all the nodes in the cluster simultaneously and restart them, as shown in the following command from the MinIO client (mc):

mc admin update <minio alias, e.g., myminio>

NOTE: some releases might not allow rolling upgrades, this is always called out in the release notes and it is generally advised to read release notes before upgrading. In such a situation mc admin update is the recommended upgrading mechanism to upgrade all servers at once.

Important things to remember during MinIO upgrades

  • mc admin update will only work if the user running MinIO has write access to the parent directory where the binary is located, for example if the current binary is at /usr/local/bin/minio, you would need write access to /usr/local/bin.
  • mc admin update updates and restarts all servers simultaneously, applications would retry and continue their respective operations upon upgrade.
  • mc admin update is disabled in kubernetes/container environments, container environments provide their own mechanisms to rollout of updates.
  • In the case of federated setups mc admin update should be run against each cluster individually. Avoid updating mc to any new releases until all clusters have been successfully updated.
  • If using kes as KMS with MinIO, just replace the binary and restart kes more information about kes can be found here
  • If using Vault as KMS with MinIO, ensure you have followed the Vault upgrade procedure outlined here: https://www.vaultproject.io/docs/upgrading/index.html
  • If using etcd with MinIO for the federation, ensure you have followed the etcd upgrade procedure outlined here: https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrading-etcd.md

Explore Further

Contribute to MinIO Project

Please follow MinIO Contributor's Guide

License

Use of MinIO is governed by the Apache 2.0 License found at LICENSE.

Comments
  • I/O Timeout

    I/O Timeout

    Howdy folks,

    No matter running the latest version or not, or latest Debian (9) or not, on RAID 10, with 30 transfers and full verbose, we get:

    ERRO[0124] Unable to create object part. cause=read tcp 46.4..:443->89.114.13*.*:49244: i/o timeout source=[object-handlers.go:817:objectAPIHandlers.PutObjectPartHandler()] stack=fs-v1-helpers.go:272:fsCreateFile fs-v1-multipart.go:523:fsObjects.PutObjectPart :339:(*fsObjects).PutObjectPart object-handlers.go:814:objectAPIHandlers.PutObjectPartHandler api-router.go:46:(objectAPIHandlers).PutObjectPartHandler-fm

    Any hint?

    Disks are fine, plenty of IO available. And these are mostly EPS files, jpg, etc, only 63GB total. I got this very same error in two servers.

    priority: medium 
    opened by xmovu 89
  • Error : hash does not match

    Error : hash does not match

    after updating minio to 2021-05-27T22:06:31Z we got for alot of files such a message in the console

    minio04 API: SYSTEM() minio04 Time: 10:27:54 UTC 06/05/2021 minio04 DeploymentID: 7c507082-fe65-439a-9391-3b44b1b1d2ed minio04 Error: Disk: http://minio01:9000/drive6 -> alwaqiyah/Cong_Sweden_Fiter1441SD.mp4/aed31203-8a34-4a28-92dc-063bfe0dc34c/part.1 - content hash does not match - expected c5e2c21880064196b4c8c21bad1b08452ba8662ae32408d92343086a34412322, got 73462c13ceab3d7d4e1849af26b94fa716ad868d9080344999a61e6ab9bc5510 (*errors.errorString) minio04 2: cmd/bitrot-streaming.go:179:cmd.(*streamingBitrotReader).ReadAt() minio04 1: cmd/erasure-decode.go:163:cmd.(*parallelReader).Read.func1()

    minio04 API: SYSTEM() minio04 Time: 10:27:54 UTC 06/05/2021 minio04 DeploymentID: 7c507082-fe65-439a-9391-3b44b1b1d2ed minio04 Error: Disk: http://minio03:9000/drive6 -> alwaqiyah/Cong_Sweden_Fiter1441SD.mp4/aed31203-8a34-4a28-92dc-063bfe0dc34c/part.1 - content hash does not match - expected 6319f76775b13b6f23495b4e53374f7629ce37ce27ccb971ae42bed55efe5771, got e271da0633bf2c805eb576bf3b7379013b27d8621ad066c015de7477cde0ced0 (*errors.errorString) minio04 2: cmd/bitrot-streaming.go:179:cmd.(*streamingBitrotReader).ReadAt() minio04 1: cmd/erasure-decode.go:163:cmd.(*parallelReader).Read.func1()

    Current Behavior

    in the browser we got something like <Error><Code>SlowDown</Code><Message>Resource requested is unreadable, please reduce your request rate</Message><Key>Cong_Sweden_Fiter1441SD.mp4</Key><BucketName>alwaqiyah</BucketName><Resource>/Cong_Sweden_Fiter1441SD.mp4</Resource><RequestId>1685A99F3532F870</RequestId><HostId>7c507082-fe65-439a-9391-3b44b1b1d2ed</HostId></Error>

    server enviroment

    • Version 2021-05-27T22:06:31Z
    • Server setup and configuration: 4 nodes CPU : Intel(R) Xeon(R) CPU D-1541 @ 2.10GHz 8C /16T MEMORY: 32G Disk: 6 disk 11T per disk
    • Operating System and version (uname -a): Linux minio04 5.4.0-73-generic #82-Ubuntu SMP Wed Apr 14 17:39:42 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
    community 
    opened by mazataza 74
  • Minio server stops in every few hours

    Minio server stops in every few hours

    Expected Behavior

    Expected behavior is for Server to remain up till it is not stopped.

    Current Behavior

    Server stops(process dies) in every few hours.

    Steps to Reproduce (for bugs)

    1. docker pull minio/minio:edge
    2. docker run -p 9000:9000 minio/minio:edge server /export
    3. Wait for few hours and the Server would go down

    Context

    Because of this issue, I need to restart the Server in every few hours. Then I restart it and a new public and private key is generated. Then I need to change the keys in my code and redeploy it. A new Server instance is created every time I restart it, which essentially means that all my files, stored in previous Server instance, get lost.

    Your Environment

    • Server type and version: Latest from Master branch
    • Operating System and version: centos-release-7-3.1611.el7.centos.x86_64
    community 
    opened by puneetpandey37 59
  • Minio cluster fails to start/sync

    Minio cluster fails to start/sync "Waiting for a minimum of 2 disks to come online"

    Expected Behavior

    I expect it to start or at least provide log data as to what is wrong.

    Current Behavior

    All 4 nodes show the same data in the log.

    Waiting for a minimum of 2 disks to come online (elapsed 16m31s) Waiting for a minimum of 2 disks to come online (elapsed 16m32s) Waiting for a minimum of 2 disks to come online (elapsed 16m33s) Waiting for a minimum of 2 disks to come online (elapsed 16m34s) Waiting for a minimum of 2 disks to come online (elapsed 16m35s)

    If I log into a container I can ping the other containers and a curl to the http://minio#:9000/data produces the following

    <Error><Code>XMinioServerNotInitialized</Code><Message>Server not initialized, please try again.</Message><BucketName>data</BucketName><Resource>/data</Resource><RequestId>15F7158280B18599</RequestId><HostId></HostId></Error>/

    Steps to Reproduce (for bugs)

    Note sure how to reproduce but it happened when my docker swarm failed and I had to recreate it. While digging around in the config for minio, stored in the volume I noticed that recreating the stack updated to a newer minio so the old version was RELEASE.2019-09-26T19-42-35Z, while the new one is RELEASE.2020-02-20T22-51-23Z.

    Note: This had run for months without issue.

    Your Environment

    • Version used (minio version): RELEASE.2020-02-20T22-51-23Z
    • Environment name and version (e.g. nginx 1.9.1): Docker 19.03.6
    • Server type and version: 20 VM Docker Swarm
    • Operating System and version (uname -a): (docker nodes are) CentOS Linux release 7.7.1908 (Core)

    I've been working on this for about a week, digging through posts to see if I can find a way to get it back up and have not found anything. So I thought I would ask here.

    For Reference, this is my stack.yaml

    version: "3.2"
    services:
      minio1:
        image: minio/minio
        volumes:
          - data2:/data
        networks:
          - traefik_default
          - minio
        deploy:
           labels:
              - "traefik.frontend.rule=Host:minio-c.docker.mydomain.com"
              - "traefik.port=9000"
              - "traefik.enable=true"     
        environment:
          MINIO_ACCESS_KEY: <MY ACCESS KEY>
          MINIO_SECRET_KEY: <MY SECRET KEY>
        command: server  http://minio1:9000/data http://minio4:9000/data http://minio2:9000/data http://minio3:9000/data
    
      minio2:
        image: minio/minio
        volumes:
          - data3:/data
        networks:
          - traefik_default
          - minio
        deploy:
           labels:
              - "traefik.frontend.rule=Host:minio-c.docker.mydomain.com"
              - "traefik.port=9000"
              - "traefik.enable=true"     
        environment:
          MINIO_ACCESS_KEY: <MY ACCESS KEY>
          MINIO_SECRET_KEY: <MY SECRET KEY>
        command: server http://minio2:9000/data http://minio4:9000/data http://minio1:9000/data http://minio3:9000/data
    
      minio3:
        image: minio/minio
        volumes:
          - data4:/data
        networks:
          - traefik_default
          - minio
        deploy:
           labels:
              - "traefik.frontend.rule=Host:minio-c.docker.mydomain.com"
              - "traefik.port=9000"
              - "traefik.enable=true"     
        environment:
          MINIO_ACCESS_KEY: <MY ACCESS KEY>
          MINIO_SECRET_KEY: <MY SECRET KEY>
        command: server http://minio3:9000/data http://minio4:9000/data http://minio1:9000/data http://minio2:9000/data 
    
      minio4:
        image: minio/minio
        volumes:
          - data5:/data
        networks:
          - traefik_default
          - minio
        deploy:
           labels:
              - "traefik.frontend.rule=Host:minio-c.docker.mydomain.com"
              - "traefik.port=9000"
              - "traefik.enable=true"     
        environment:
          MINIO_ACCESS_KEY: <MY ACCESS KEY>
          MINIO_SECRET_KEY: <MY SECRET KEY>
        command: server http://minio4:9000/data http://minio1:9000/data http://minio2:9000/data http://minio3:9000/data
        
    <Removed Volumes>
    <Removed Networks>
    
    
    working as intended community 
    opened by Aphris-Karu 58
  • Implement S3 Gateway to third party cloud storage providers.

    Implement S3 Gateway to third party cloud storage providers.

    Description

    Currently supported backend is Azure Blob Storage.

    export MINIO_ACCESS_KEY=azureaccountname
    export MINIO_SECRET_KEY=azureaccountkey
    minio gateway azure
    

    Motivation and Context

    Minio gateway adds Amazon S3 compatibility to third party cloud storage providers.

    How Has This Been Tested?

    Manually and by rest of the @minio/core

    Types of changes

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [x] New feature (non-breaking change which adds functionality)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)

    Checklist:

    • [x] My code follows the code style of this project.
    • [x] My change requires a change to the documentation.
    • [x] I have updated the documentation accordingly.
    • [x] I have added tests to cover my changes.
    • [x] All new and existing tests passed.
    opened by krishnasrinivas 57
  • WebUI/listObjects() show/return object. Download/getObject() return empty file / cause error

    WebUI/listObjects() show/return object. Download/getObject() return empty file / cause error "NoSuchKey"

    The WebUI displays a file with a plausible size of 740B. Screenshot 2022-06-05 144902

    Downloading this file via WebUI results in an empty file: Screenshot 2022-06-05 145130

    API access by way of listObjects() returns an Iterator<Result> results, which includes the above mentioned objectName.

    Calling getObject() with the objectName from above causes an ErrorResponseException with an errorResponseCode "NoSuchKey".

    Expected Behavior

    Parts of Minio seem to think the file exists, others don't. A displayed / listed objectName should be retrievable.

    Current Behavior

    See description.

    Trace of a download attempt by WebUI: Screenshot 2022-06-05 150522

    Context

    Usecase is a regular low intensity consistency check with a cassandra db. We traverse all Minio objects and check if cassandra is up-to-date.

    Regression

    na

    Your Environment

    MinIO VERSION 2022-04-16T04:26:02Z 4 dedicated Rasp-4B with 8GB each with dedicated 2TB SSD. Linux rasp-3 5.15.32-v8+ #1538 SMP PREEMPT Thu Mar 31 19:40:39 BST 2022 aarch64 GNU/Linux

    This has been working like a charm for about a year with > 16 Mio object and 2 TB data.

    community priority: medium 
    opened by RAlfoeldi 55
  • Hadoop 3.3 Compatibility issue with single drive mode (minio server /data)

    Hadoop 3.3 Compatibility issue with single drive mode (minio server /data)

    MinIO works with Hadoop 3.2, but not with Hadoop 3.3.

    Current Behavior

    Repro code:

    val conf = new Configuration()
    conf.set("fs.s3a.endpoint", "http://127.0.0.1:9000")
    conf.set("fs.s3a.path.style.access", "true")
    conf.set("fs.s3a.access.key", "user_access_key")
    conf.set("fs.s3a.secret.key", "password")
    
    val path = new Path("s3a://comcast-test")
    val fs = path.getFileSystem(conf)
    
    fs.mkdirs(new Path("/testdelta/_delta_log"))
    fs.getFileStatus(new Path("/testdelta/_delta_log"))
    

    Fails with FileNotFoundException. The same code works in real S3. It also works in Hadoop 3.2. Only fails on 3.3 and newer Hadoop branches.

    Possible Solution

    This works in Hadoop 3.2 because of this infamous "Is this necessary?" block of code https://github.com/apache/hadoop/blob/branch-3.2.0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L2204-L2223

    that was removed in Hadoop 3.3 - https://github.com/apache/hadoop/blob/branch-3.3.0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L2179

    and this causes the regression

    Steps to Reproduce (for bugs)

    See code above.

    Context

    Some applications need to create subdirectories before they can write to Minio, so this affects

    Regression

    Yes, this is a regression.

    Your Environment

    • Subnet ticket 3439
    community fixed 
    opened by Tagar 55
  • AWS iOS SDK working with minio

    AWS iOS SDK working with minio

    Error messgae

    SignatureDoesNotMatchThe request signature we calculated does not match the signature you provided. Check your key and signing method./images/test.jpg3L1373L137 Upload failed with error: (The operation couldn’t be completed. (com.amazonaws.AWSServiceErrorDomain error 3.))

    Here is how I configure

    let accessKey = "xxxxxxx"
    let secretKey = "xxxxxxx"       
    let credentialsProvider = AWSStaticCredentialsProvider(accessKey: accessKey, secretKey: secretKey)
    let configuration = AWSServiceConfiguration(region: AWSRegionType.USEast1, endpoint: AWSEndpoint(region: .USEast1, service: .APIGateway, url: URL( string:"http://xxxx.com:9000")),credentialsProvider: credentialsProvider)
    AWSServiceManager.default().defaultServiceConfiguration = configuration
    

    Possible Solution

    refer to this issue, https://github.com/minio/mc/issues/1707 maybe it caused by region issue?

    Your Environment

    • Version used:

      • official docker latest
      • awss3 lib used: pod 'AWSS3', '~> 2.5'
    community priority: high 
    opened by atom2ueki 54
  • ecosystem: Validate all spark and hadoop supported s3 connectors

    ecosystem: Validate all spark and hadoop supported s3 connectors

    Expected behaviour

    I want to use spark-shell (v2.0.1) to connect to minio local to read input data and write back output data.

    Actual behaviour

    Attempting a simple read of a text file (json in this case) results in the error:

    com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 400, AWS Service: Amazon S3, AWS Request ID: null, AWS Error Code: null, AWS Error Message: Bad Request
    

    (full stack will be below)

    Steps to reproduce the behaviour

    Install Oracle Java 8 (1.8.0_71) Install Scala 2.11.8 Minio installed by Home Brew Install Spark 2.0.1

    Start the minio server local, log into the browser UI, create a bucket named data and upload a small json file to it (e.g. bacon_tiny.json).

    { "bacon": {
        "isCrispy": true,
        "isDelicious": true,
        "isVegan": false
        }
    }
    

    Relevant bits of spark-env.sh (all other bits left to default):

    HADOOP_CONF_DIR=/Users/bkarels/hadoop/hadoop-2.7.3/etc/hadoop
    SPARK_LOCAL_IP="127.0.0.1"
    SPARK_MASTER_HOST="localhost"
    SPARK_MASTER_WEBUI_PORT=8080
    SPARK_MASTER_PORT=7077
    SPARK_DAEMON_JAVA_OPTS="-Djava.net.preferIPv4Stack=true -Dcom.amazonaws.services.s3.enableV4=true"
    

    core-site.xml (from ~/hadoop/hadoop-2.7.3/etc/hadoop):

    <?xml version="1.0" encoding="UTF-8"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    
    <configuration>
      <property>
        <name>fs.s3a.endpoint</name>
        <value>http://127.0.0.1:9000</value>
      </property>
    
      <property>
        <name>fs.s3a.access.key</name>
        <description>AWS access key ID.</description>
        <value>*************************</value>
      </property>
    
      <property>
        <name>fs.s3a.secret.key</name>
        <description>AWS secret key.</description>
        <value>*********************************************</value>
      </property>
    
    </configuration>
    

    Starting the spark-shell

    $ ./bin/spark-shell --master local[4] --jars "./bin/hadoop-aws-2.7.1.jar,./bin/aws-java-sdk-1.7.4.jar"
    scala> val bacon = sc.textFile("s3a://data/bacon_tiny.json").first
    
    com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 400, AWS Service: Amazon S3, AWS Request ID: null, AWS Error Code: null, AWS Error Message: Bad Request
      at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:798)
      at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:421)
      at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232)
      at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528)
      at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1031)
      at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:994)
      at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:297)
      at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
      at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
      at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
      at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
      at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
      at org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:258)
      at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:229)
      at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
      at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:199)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:248)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:246)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.rdd.RDD.partitions(RDD.scala:246)
      at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:248)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:246)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.rdd.RDD.partitions(RDD.scala:246)
      at org.apache.spark.rdd.RDD$$anonfun$take$1.apply(RDD.scala:1303)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
      at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
      at org.apache.spark.rdd.RDD.take(RDD.scala:1298)
      at org.apache.spark.rdd.RDD$$anonfun$first$1.apply(RDD.scala:1338)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
      at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
      at org.apache.spark.rdd.RDD.first(RDD.scala:1337)
      ... 48 elided
    

    All else being equal, I can adjust core-site.xml to point to an actual s3 instance and things work which is what has me thinking this might be a Minio issue.

    Minio version

    Version: 2016-09-11T17:42:18Z
    Release-Tag: RELEASE.2016-09-11T17-42-18Z
    Commit-ID: 85e2d886bcb005d49f3876d6849a2b5a55e03cd3
    

    System information

    Running on OSX 10.11.6
    MacBook Pro (Retina, 15-inch, Mid 2015)
    2.5 GHz Intel Core i7
    16 GB 1600 MHz DDR3
    
    community fixed priority: medium 
    opened by bradkarels 54
  • ARM64 : Minio server crashes randomly

    ARM64 : Minio server crashes randomly

    Minio server is terminating randomly on ARM64 platform. Built the latest code on ARM64 platform and used for testing.The deployment type is "distributed mode". In a week testing on 6 nodes, 3 servers crashed. The load was moderate when these crashes happened.

    Log messages ::

    goroutine 338370336 [IO wait]: runtime.gopark(0x1821768, 0xfffe25850a98, 0x4001841b02, 0x5) /usr/local/go/src/runtime/proc.go:301 +0xf0 fp=0x40042fc9c0 sp=0x40042fc9a0 pc=0x43cf50 runtime.netpollblock(0xfffe25850a70, 0x72, 0x409266a000) /usr/local/go/src/runtime/netpoll.go:389 +0xa4 fp=0x40042fca00 sp=0x40042fc9c0 pc=0x4385f4 internal/poll.runtime_pollWait(0xfffe25850a70, 0x72, 0xffffffffffffffff) /usr/local/go/src/runtime/netpoll.go:182 +0x48 fp=0x40042fca30 sp=0x40042fca00 pc=0x437b48 internal/poll.(*pollDesc).wait(0x408b030818, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0xa0 fp=0x40042fca60 sp=0x40042fca30 pc=0x493fc0 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(*FD).Read(0x408b030800, 0x409266a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) /usr/local/go/src/internal/poll/fd_unix.go:169 +0x170 fp=0x40042fcac0 sp=0x40042fca60 pc=0x494e40 net.(*netFD).Read(0x408b030800, 0x409266a000, 0x1000, 0x1000, 0x40042fcb78, 0x413008, 0x40042fcb60) /usr/local/go/src/net/fd_unix.go:202 +0x44 fp=0x40042fcb20 sp=0x40042fcac0 pc=0x598c14 net.(*conn).Read(0x4031f50158, 0x409266a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) /usr/local/go/src/net/net.go:177 +0x5c fp=0x40042fcb80 sp=0x40042fcb20 pc=0x5ab42c net/http.(*persistConn).Read(0x408dc807e0, 0x409266a000, 0x1000, 0x1000, 0x40042fcc68, 0x6c2f30, 0x405fa8c840) /usr/local/go/src/net/http/transport.go:1524 +0x60 fp=0x40042fcc00 sp=0x40042fcb80 pc=0x6c1da0 bufio.(*Reader).fill(0x4004105a40) /usr/local/go/src/bufio/bufio.go:100 +0x100 fp=0x40042fcc50 sp=0x40042fcc00 pc=0x4b0000 bufio.(*Reader).Peek(0x4004105a40, 0x1, 0x0, 0x0, 0x1, 0x400d6c5600, 0x0) /usr/local/go/src/bufio/bufio.go:138 +0x40 fp=0x40042fcc70 sp=0x40042fcc50 pc=0x4b0180 net/http.(*persistConn).readLoop(0x408dc807e0) /usr/local/go/src/net/http/transport.go:1677 +0x16c fp=0x40042fcfd0 sp=0x40042fcc70 pc=0x6c282c runtime.goexit() /usr/local/go/src/runtime/asm_arm64.s:1128 +0x4 fp=0x40042fcfd0 sp=0x40042fcfd0 pc=0x46aa74 created by net/http.(*Transport).dialConn /usr/local/go/src/net/http/transport

    Expected Behavior

    Minio server shouldn't terminate.

    Current Behavior

    Minio server terminates.

    Your Environment

    • Version used (minio version): Minio (built from source on ARM64 platform) Version: DEVELOPMENT.GOGET Release-Tag: DEVELOPMENT.GOGET Commit-ID: DEVELOPMENT.GOGET

    • Operating System and version (uname -a): aarch64 aarch64 aarch64 GNU/Linux

    working as intended community priority: medium 
    opened by kattakarthik 53
  • Scheduled removal of MinIO Gateway for GCS, Azure, HDFS

    Scheduled removal of MinIO Gateway for GCS, Azure, HDFS

    MinIO Gateway will be removed by June 1st, 2022 from the MinIO repository:

    Community Users

    • Please migrate your MinIO Gateway deployments from Azure, GCS, HDFS to MinIO Distributed Setups

    • MinIO S3 Gateway will be renamed as "minio edge" and will only support MinIO Backends to extend the functionality of supporting remote credentials etc locally as "read-only" for authentication and policy management.

    • Newer MinIO NAS/Single drive setups will move to single data and 0 parity mode (that will re-purpose the erasure-coded backend used for distributed setups but with 0 parity). This would allow for distributed setup features to be available for single drive deployments as well such as

      • Versioning
      • ILM
      • Replication and more...
    • Existing setups for NAS/Single drive setups will work as-is nothing changes.

    Paid Users

    All existing paid customers will be supported as per their LTS support contract. If there are bugs they will be fixed and backported fixes will be provided. No new features will be implemented for Gateway implementations.

    community fixed priority: high 
    opened by harshavardhana 50
  • move to klauspost/compress@master until new release

    move to klauspost/[email protected] until new release

    Description

    move to klauspost/[email protected] until new release

    Motivation and Context

    remove forked replace directive

    How to test this PR?

    nothing should change or break

    Types of changes

    • [x] Bug fix (non-breaking change which fixes an issue)
    • [ ] New feature (non-breaking change which adds functionality)
    • [ ] Optimization (provides speedup with no functional changes)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)

    Checklist:

    • [ ] Fixes a regression (If yes, please add commit-id or PR # here)
    • [ ] Unit tests added/updated
    • [ ] Internal documentation updated
    • [ ] Create a documentation update request here
    opened by harshavardhana 4
  • vectorize cluster-wide calls such as bucket operations

    vectorize cluster-wide calls such as bucket operations

    Description

    vectorize cluster-wide calls such as bucket operations

    Motivation and Context

    This PR is a re-implementation of the following calls

    • MakeBucket()
    • DeleteBucket()
    • GetBucketInfo()

    into peer-level calls, this is needed to ensure that we start reducing the number of connections needed to perform bucket operations - that are always cluster-wide, with a high number of disk setups, it is important that we reduce the state of open connections between hosts.

    New connections incur costs, that can in turn put a lot of syscall() load on the system, this PR should reduce the number of connections by order of magnitude.

    How to test this PR?

    This a fundamental change to an existing functionality and must require testing of corner cases thoroughly.

    Types of changes

    • [x] Bug fix (non-breaking change which fixes an issue)
    • [ ] New feature (non-breaking change which adds functionality)
    • [x] Optimization (provides speedup with no functional changes)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)

    Checklist:

    • [ ] Fixes a regression (If yes, please add commit-id or PR # here)
    • [x] Unit tests added/updated
    • [ ] Internal documentation updated
    • [ ] Create a documentation update request here
    do-not-close 
    opened by harshavardhana 2
  • IPv6 is attempted to use even if not supported

    IPv6 is attempted to use even if not supported

    Expected Behavior

    No IPv6 shouldn't be used if system doesn't support it. It should be especially true if I hardcode addresses from command line. Why it should even attempt to bind ::1 ?

    Current Behavior

    > ./minio server --address "127.0.0.1:9000" --console-address "127.0.0.1:9090" /miniodata
    
    API: SYSTEM()
    Time: 09:30:19 UTC 12/23/2022
    Error: listen tcp [::1]:9000: bind: address family not supported by protocol (*net.OpError)
           2: internal/logger/logger.go:258:logger.LogIf()
           1: cmd/signals.go:79:cmd.handleSignals()
    >
    

    Your Environment

    > ./minio --version
    minio version RELEASE.2022-12-12T19-27-27Z (commit-id=a469e6768df4d5d2cb340749fa58e4721a7dee96)
    Runtime: go1.19.4 linux/amd64
    License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
    Copyright: 2015-2022 MinIO, Inc.
    > uname -sr
    Linux 5.15.72
    >
    
    community triage 
    opened by hasso 1
  • Deduplicate erasure sets to remove uploads from

    Deduplicate erasure sets to remove uploads from

    Description

    Untested.

    Motivation and Context

    How to test this PR?

    Types of changes

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [ ] New feature (non-breaking change which adds functionality)
    • [ ] Optimization (provides speedup with no functional changes)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)

    Checklist:

    • [ ] Fixes a regression (If yes, please add commit-id or PR # here)
    • [ ] Unit tests added/updated
    • [ ] Internal documentation updated
    • [ ] Create a documentation update request here
    opened by krisis 2
  • Add loong64 support

    Add loong64 support

    Description

    Add support for loong64. At present, the support of loong64 has been added in the golang source code.

    Motivation and Context

    The LoongArch architecture (LoongArch) is an Instruction Set Architecture (ISA) that has a RISC style.

    Documentations: ISA: https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html ABI: https://loongson.github.io/LoongArch-Documentation/LoongArch-ELF-ABI-EN.html More docs can be found at: https://loongson.github.io/LoongArch-Documentation/README-EN.html

    How to test this PR?

    ./buildscripts/checkdeps.sh ./buildscripts/cross-compile.sh

    When performing cross-compilation, rely on 2 PRs: https://github.com/minio/madmin-go/pull/163 https://github.com/shirou/gopsutil/pull/1228

    Types of changes

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [ ] New feature (non-breaking change which adds functionality)
    • [ ] Optimization (provides speedup with no functional changes)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)

    Checklist:

    • [ ] Fixes a regression (If yes, please add commit-id or PR # here)
    • [ ] Unit tests added/updated
    • [ ] Internal documentation updated
    • [ ] Create a documentation update request here
    opened by zhaixiaojuan 3
  • decom: List pool after decom and error out if an object is found

    decom: List pool after decom and error out if an object is found

    Description

    As a sanity check measurement, add a listing of the pool after decomissiing and try if it can find any object, mark decomissionin status as failed if that is the case

    Motivation and Context

    How to test this PR?

    Types of changes

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [ ] New feature (non-breaking change which adds functionality)
    • [ ] Optimization (provides speedup with no functional changes)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)

    Checklist:

    • [ ] Fixes a regression (If yes, please add commit-id or PR # here)
    • [ ] Unit tests added/updated
    • [ ] Internal documentation updated
    • [ ] Create a documentation update request here
    opened by vadmeste 1
Releases(RELEASE.2023-01-02T09-40-09Z)
  • RELEASE.2023-01-02T09-40-09Z(Jan 3, 2023)

    What's Changed

    • feat: allow listening specific addrs for API port by @harshavardhana in https://github.com/minio/minio/pull/16223
    • Refactor HTTP transports by @donatello in https://github.com/minio/minio/pull/16222
    • log: Do not interpret verbs in object names in console output by @vadmeste in https://github.com/minio/minio/pull/16233
    • Implement STS account info API by @vadmeste in https://github.com/minio/minio/pull/16115
    • Add endpoints for managing IAM policies by @taran-p in https://github.com/minio/minio/pull/15897
    • Add HTTP2 config option for policy plugin by @donatello in https://github.com/minio/minio/pull/16225
    • fix: rebalance to account for object's on-disk size by @krisis in https://github.com/minio/minio/pull/16240
    • do not log checksum mismatch error, client received the error by @harshavardhana in https://github.com/minio/minio/pull/16246
    • Cleanup remote targets automatically on replication config removal. by @poornas in https://github.com/minio/minio/pull/16221
    • fix: support event name s3:Restore:* by @harshavardhana in https://github.com/minio/minio/pull/16257
    • fix: remove scanner healing with unnecessary logs by @harshavardhana in https://github.com/minio/minio/pull/16260
    • fix: remove unnecessary logs for client conn errors by @harshavardhana in https://github.com/minio/minio/pull/16261
    • Faster startup of large distributed systems with latency by @klauspost in https://github.com/minio/minio/pull/16259
    • use newDynamicTimeoutWithOpts() when appropriate by @harshavardhana in https://github.com/minio/minio/pull/16266
    • Do not return an error when version disparity is detected by @vadmeste in https://github.com/minio/minio/pull/16269
    • helm: add policy to svcacct by @orblazer in https://github.com/minio/minio/pull/16272
    • helm: Fix post job template by @mruzicka in https://github.com/minio/minio/pull/16236
    • Add API to fetch policy user/group associations by @taran-p in https://github.com/minio/minio/pull/16239
    • Avoid config migration on every startup by @klauspost in https://github.com/minio/minio/pull/16278
    • Do not save credentials in config.json by @vadmeste in https://github.com/minio/minio/pull/16275
    • allow retries for transaction lock on startup by @harshavardhana in https://github.com/minio/minio/pull/16273
    • serialize replication stats() only when needed by @harshavardhana in https://github.com/minio/minio/pull/16280
    • trace: Add bootstrap tracing events by @vadmeste in https://github.com/minio/minio/pull/16286
    • Fix DeleteObject when only free versions remain by @krisis in https://github.com/minio/minio/pull/16289
    • update console v0.22.3 by @harshavardhana in https://github.com/minio/minio/pull/16292
    • Remove site replication config if it succeeded locally by @poornas in https://github.com/minio/minio/pull/16279
    • tier verification leaks fd, that must be closed by @jiuker in https://github.com/minio/minio/pull/16296
    • upgrade UBI image to 8.7 by @harshavardhana in https://github.com/minio/minio/pull/16301
    • persist replication stats with leader lock by @poornas in https://github.com/minio/minio/pull/16282
    • avoid locks for internal and invalid buckets in MakeBucket() by @harshavardhana in https://github.com/minio/minio/pull/16302
    • helm: disallow empty containers in post job template by @chamburr in https://github.com/minio/minio/pull/16281
    • Validate replication target update to avoid duplicate endpoints by @poornas in https://github.com/minio/minio/pull/16311
    • debug: Add X-Amz-Request-ID to lock/unlock calls by @vadmeste in https://github.com/minio/minio/pull/16309
    • allow locks with object affinity to spread across pools by @harshavardhana in https://github.com/minio/minio/pull/16312
    • metrics: Show healing info for all nodes by @vadmeste in https://github.com/minio/minio/pull/16315
    • helm: Make bucket existence check faster by @wokalski in https://github.com/minio/minio/pull/16321
    • fix: DeleteObject() API with versionId under replication by @harshavardhana in https://github.com/minio/minio/pull/16325
    • fix: a deadlock by refactoring listBuckets() under site replication by @harshavardhana in https://github.com/minio/minio/pull/16323
    • doc: Update prometheus metrics list by @vadmeste in https://github.com/minio/minio/pull/16329
    • add missing fields in audit logs for non-compressed handlers by @harshavardhana in https://github.com/minio/minio/pull/16328
    • disable builds for go1.18 by @harshavardhana in https://github.com/minio/minio/pull/16332
    • Fix misplaced write response command in DetachPolicyBuiltin by @taran-p in https://github.com/minio/minio/pull/16333
    • fix: tweak read buffer size to reduce over-reading by @klauspost in https://github.com/minio/minio/pull/16338
    • fix: close helmInfo file descriptor by @jiuker in https://github.com/minio/minio/pull/16319
    • remove the startup jitter before locks() by @harshavardhana in https://github.com/minio/minio/pull/16340

    New Contributors

    • @orblazer made their first contribution in https://github.com/minio/minio/pull/16272
    • @mruzicka made their first contribution in https://github.com/minio/minio/pull/16236
    • @chamburr made their first contribution in https://github.com/minio/minio/pull/16281
    • @wokalski made their first contribution in https://github.com/minio/minio/pull/16321

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-12-12T19-27-27Z...RELEASE.2023-01-02T09-40-09Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-12-12T19-27-27Z(Dec 13, 2022)

    What's New

    • LDAP configuration now supports enabling DNS SRV record lookup to find LDAP servers #16201
    • New API to associate/dissociate policies with LDAP users or groups #16182

    What's Changed

    • remove unnecessary logs for truncated XML inputs by @harshavardhana in https://github.com/minio/minio/pull/16184
    • Encrypt checksums with KMS on CompleteMultipartUpload by @klauspost in https://github.com/minio/minio/pull/16177
    • feat: Encrypt s3zip file index by @klauspost in https://github.com/minio/minio/pull/16179
    • reuse sha256 in config GetSettings by @jiuker in https://github.com/minio/minio/pull/16188
    • Add version id to healresult by @klauspost in https://github.com/minio/minio/pull/16193
    • Replace filepathx with fork by @klauspost in https://github.com/minio/minio/pull/16192
    • fix: translate tier add errors properly by @harshavardhana in https://github.com/minio/minio/pull/16191
    • remove hard limit for number of buckets by @harshavardhana in https://github.com/minio/minio/pull/16194
    • fix: delete marker discrepancies via DeleteObject() API by @harshavardhana in https://github.com/minio/minio/pull/16195
    • Add IAM API to attach/detach policies for LDAP by @donatello in https://github.com/minio/minio/pull/16182
    • avoid double deletes() when no more versions by @harshavardhana in https://github.com/minio/minio/pull/16206
    • do not rename multipart failed transactions back to tmp by @harshavardhana in https://github.com/minio/minio/pull/16204
    • Use different subnet public key during dev/test by @anjalshireesh in https://github.com/minio/minio/pull/16216
    • fix: iso8601TimeFormat padding issue for certain nanoseconds by @harshavardhana in https://github.com/minio/minio/pull/16207
    • Add LDAP DNS SRV record lookup support by @donatello in https://github.com/minio/minio/pull/16201

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-12-07T00-56-37Z...RELEASE.2022-12-12T19-27-27Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-12-07T00-56-37Z(Dec 7, 2022)

    What's Changed

    • fix: in disk cache readCacheFileStream should closed upon return by @jiuker in https://github.com/minio/minio/pull/16138
    • Start PR on Enterprise when there is new MinIO Version by @cniackz in https://github.com/minio/minio/pull/16121
    • kms: add support for KES enclaves by @aead in https://github.com/minio/minio/pull/16139
    • fix: update the JSON keys for latest 'mc' release by @harshavardhana in https://github.com/minio/minio/pull/16171
    • Run staticcheck on CI by @klauspost in https://github.com/minio/minio/pull/16170
    • opt: Only stream big data usage caches by @klauspost in https://github.com/minio/minio/pull/16168
    • Avoid hot loop when lock is cancelled by @klauspost in https://github.com/minio/minio/pull/16169
    • avoid logging gzipped body in trace output by @harshavardhana in https://github.com/minio/minio/pull/16172
    • Add minimal setup command to helm chart's readme by @rebrendov in https://github.com/minio/minio/pull/16165
    • helm: modify the job create order by @dormanze in https://github.com/minio/minio/pull/15696
    • s3: Return correct error when Version is invalid in policy document by @vadmeste in https://github.com/minio/minio/pull/16178
    • use readlock instead of writelock to get heal information by @jiuker in https://github.com/minio/minio/pull/16175
    • Remove go1.18 from Github workflow tests by @vadmeste in https://github.com/minio/minio/pull/16180
    • Add X-Amz-Request-Id to internode calls by @vadmeste in https://github.com/minio/minio/pull/16146
    • xl: Delete older data-dir when replacing an existing version-id by @vadmeste in https://github.com/minio/minio/pull/16176
    • Bump up madmin-go to v2 by @donatello in https://github.com/minio/minio/pull/16162
    • Populate end_session_endpoint by @reivaj05 in https://github.com/minio/minio/pull/16183

    New Contributors

    • @rebrendov made their first contribution in https://github.com/minio/minio/pull/16165

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-12-02T19-19-22Z...RELEASE.2022-12-07T00-56-37Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-12-02T19-19-22Z(Dec 2, 2022)

    What's Changed

    • update reedsolomon v1.11.3 by @harshavardhana in https://github.com/minio/minio/pull/16149
    • re-order the top-level config keys for priority by @harshavardhana in https://github.com/minio/minio/pull/16150
    • repl: resync orchestrator to use global shared lock by @vadmeste in https://github.com/minio/minio/pull/16154
    • trace: Publish storage layer errors by @vadmeste in https://github.com/minio/minio/pull/16153
    • re-implement StorageInfo to be a peer call by @harshavardhana in https://github.com/minio/minio/pull/16155
    • Persist updated metadata by @klauspost in https://github.com/minio/minio/pull/16160
    • s3zip: Limit over-read for single file by @klauspost in https://github.com/minio/minio/pull/16161
    • fix: a crash in KMS cert reload function by @harshavardhana in https://github.com/minio/minio/pull/16158
    • ship mc along with MinIO container image by @harshavardhana in https://github.com/minio/minio/pull/16156

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-11-29T23-40-49Z...RELEASE.2022-12-02T19-19-22Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-11-29T23-40-49Z(Nov 30, 2022)

    What's Changed

    • avoid serializing decryptKey() every 15mins by @harshavardhana in https://github.com/minio/minio/pull/16135
    • kms: fix possible deadlock due to nested RLock calls. by @aead in https://github.com/minio/minio/pull/16136
    • Improvements in logger and audit webhooks by @anjalshireesh in https://github.com/minio/minio/pull/16102
    • Move http recorder to an internal library by @vadmeste in https://github.com/minio/minio/pull/16128
    • Check for abandoned data when healing by @klauspost in https://github.com/minio/minio/pull/16122
    • change dependency from amqp -> amqp091 (RabbitMQ) official by @harshavardhana in https://github.com/minio/minio/pull/16142
    • feat: Allow at most one claim based OpenID IDP by @donatello in https://github.com/minio/minio/pull/16145

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-11-26T22-43-32Z...RELEASE.2022-11-29T23-40-49Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-11-26T22-43-32Z(Nov 27, 2022)

    Highlights

    • Optimizations in Multipart code-path in multi-pool setups, via #16086
    • Fixes a regression in server side replication due to ETag mismatch issue under specific type of MinIO setups, fixed via #16129

    What's Changed

    • bucket resync: persist reset id to bucket metadata by @poornas in https://github.com/minio/minio/pull/16088
    • avoid partID lock validating uploadID exists prematurely by @harshavardhana in https://github.com/minio/minio/pull/16086
    • fix: skip free version ID and marker in metadata equality by @harshavardhana in https://github.com/minio/minio/pull/16093
    • config: Return XMinioConfigNotFound code for non existing config by @vadmeste in https://github.com/minio/minio/pull/16065
    • update single drive setup error message by @harshavardhana in https://github.com/minio/minio/pull/16098
    • Reduce temporary file clean-up waits by @klauspost in https://github.com/minio/minio/pull/16110
    • fix: checking for stale STS account under site replication by @vadmeste in https://github.com/minio/minio/pull/16109
    • fix: duplicated shared prefix with custom delimiter when listing by @klauspost in https://github.com/minio/minio/pull/16111
    • fix: Timestamp not added in diagnostics report by @anjalshireesh in https://github.com/minio/minio/pull/16114
    • notify: Return detailed err msg when connecting to target fails by @vadmeste in https://github.com/minio/minio/pull/16118
    • wrapped error will not equal by @jiuker in https://github.com/minio/minio/pull/16113
    • add disk.Close() in healFreshDisk to indicate idiomatic flow of code by @jiuker in https://github.com/minio/minio/pull/16124
    • preserve replicated ETag properly on target by @poornas in https://github.com/minio/minio/pull/16129

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-11-17T23-20-09Z...RELEASE.2022-11-26T22-43-32Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-11-17T23-20-09Z(Nov 18, 2022)

    Highlights

    • Support for full site level resync is now available for disaster recovery scenarios, refer #15753
    • Upgrade KES client dependency for v0.22.0 now supports full stateful load balancing across multiple kes endpoints, refer #16077

    What's Changed

    • helm: update bucket policy setting via 'mc anonymous' by @elg0ch0 in https://github.com/minio/minio/pull/16055
    • save deploymentID to avoid mutating request entry in Audit by @jiuker in https://github.com/minio/minio/pull/16053
    • configuring the nats target to reconnect forever by @rayjanoka in https://github.com/minio/minio/pull/16050
    • helm: fix positional parameter in template by @yanggangtony in https://github.com/minio/minio/pull/15983
    • improve server startup error when pools are incorrect by @harshavardhana in https://github.com/minio/minio/pull/16056
    • choose default values upon incorrect storage_class value by @harshavardhana in https://github.com/minio/minio/pull/16058
    • helm: add extraContainer by @tamcore in https://github.com/minio/minio/pull/15660
    • Create svcacct by @asoria-lf in https://github.com/minio/minio/pull/15939
    • errNoSuchPolicy Use errors.Is by @jiuker in https://github.com/minio/minio/pull/16062
    • site replication: Add support for site level resync by @poornas in https://github.com/minio/minio/pull/15753
    • Allow tls.certSecret in Helm chart to be templated by @OvervCW in https://github.com/minio/minio/pull/16064
    • avoid inconsistent versions healing when versions are large by @harshavardhana in https://github.com/minio/minio/pull/16066
    • Add missing argument for tpl in Helm chart (fix for bug in #16064) by @OvervCW in https://github.com/minio/minio/pull/16068
    • top-locks: Include lock-held duration by @krisis in https://github.com/minio/minio/pull/16061
    • fix: refactor getReplicateDiff for safe use by @klauspost in https://github.com/minio/minio/pull/16051
    • Send health diagnostics data as part of callhome by @anjalshireesh in https://github.com/minio/minio/pull/16006
    • update KES dependency to v0.22.0 by @aead in https://github.com/minio/minio/pull/16077
    • allow resultCh to be closed() after clusterMetaHealthInfo() by @jiuker in https://github.com/minio/minio/pull/16073
    • allow changing endpoints in distributed setups by @harshavardhana in https://github.com/minio/minio/pull/16071
    • fix: update metacache entry only once by @krisis in https://github.com/minio/minio/pull/16072
    • fix: replication autoscaling deadlock by @klauspost in https://github.com/minio/minio/pull/16084
    • distribute replication ops based on names by @klauspost in https://github.com/minio/minio/pull/16083

    New Contributors

    • @elg0ch0 made their first contribution in https://github.com/minio/minio/pull/16055
    • @tamcore made their first contribution in https://github.com/minio/minio/pull/15660
    • @asoria-lf made their first contribution in https://github.com/minio/minio/pull/15939

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-11-11T03-44-20Z...RELEASE.2022-11-17T23-20-09Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-11-11T03-44-20Z(Nov 11, 2022)

    What's Changed

    • fix: parsing multipart uploadID under site replicated setup by @poornas in https://github.com/minio/minio/pull/16048
    • netPerfRX Reset() should use write Lock() @jiuker in https://github.com/minio/minio/pull/16043

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-11-10T18-20-21Z...RELEASE.2022-11-11T03-44-20Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-11-10T18-20-21Z(Nov 10, 2022)

    What's Changed

    • rebalance status must return appropriate error initially by @harshavardhana in https://github.com/minio/minio/pull/16022
    • remove io/util for advanced golang by @yanggangtony in https://github.com/minio/minio/pull/16011
    • Use the same lock for the scanner and site replication healing by @vadmeste in https://github.com/minio/minio/pull/15985
    • Remove some logs caused by external apps by @vadmeste in https://github.com/minio/minio/pull/16027
    • remove printing map() checksums by @harshavardhana in https://github.com/minio/minio/pull/16028
    • Add notification queue metrics by @klauspost in https://github.com/minio/minio/pull/16026
    • close the reader under disk cache bitrot verification by @jiuker in https://github.com/minio/minio/pull/16024
    • ilm: Fix x-amz-expiration header evaluation by @krisis in https://github.com/minio/minio/pull/16029
    • Return remote tier for transitioned objects by @krisis in https://github.com/minio/minio/pull/16035
    • fix: avoid URL unsafe chars in multipart upload ID by @donatello in https://github.com/minio/minio/pull/16034
    • lifecycle: simplify Eval and HasActiveRules by @krisis in https://github.com/minio/minio/pull/16036
    • replication: Avoid blocking on mrf save by @poornas in https://github.com/minio/minio/pull/16045
    • Add Audit target metrics by @klauspost in https://github.com/minio/minio/pull/16044

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-11-08T05-27-07Z...RELEASE.2022-11-10T18-20-21Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-11-08T05-27-07Z(Nov 8, 2022)

    Highlights

    • Allows merging of versions across pools during Listing()
    • Inspect data API changes to v2 where all data sent is encrypted
    • Support HS256, HS384, HS512 JWT signature algo for OpenID Connect

    What's Changed

    • do not panic if webhook returns an error by @harshavardhana in https://github.com/minio/minio/pull/15970
    • ignore background healInfo in single drive setup by @harshavardhana in https://github.com/minio/minio/pull/15968
    • Clean entire tmp-old on restart by @klauspost in https://github.com/minio/minio/pull/15979
    • Update admin IDP APIs to be more RESTful by @donatello in https://github.com/minio/minio/pull/15896
    • cleanup stale parent multipart directories by @harshavardhana in https://github.com/minio/minio/pull/15980
    • config: Deprecated delay/max_wait/scanner and introduce speed by @vadmeste in https://github.com/minio/minio/pull/15941
    • converge SNSD deployments into single code by @harshavardhana in https://github.com/minio/minio/pull/15988
    • add windows port allot by "netsh dynamicport" by @yanggangtony in https://github.com/minio/minio/pull/15986
    • Implement inspect data API v2 by @vadmeste in https://github.com/minio/minio/pull/15474
    • site replication edit: validate endpoint belongs to deployment by @poornas in https://github.com/minio/minio/pull/16000
    • support HS256 series of JWT signature for OpenID connect by @harshavardhana in https://github.com/minio/minio/pull/15993
    • fix_http_body_should_be_closed by @jiuker in https://github.com/minio/minio/pull/16001
    • API is compatible with Elasticsearch 7.17.7 by @yanggangtony in https://github.com/minio/minio/pull/15992
    • Crash: Hold lock while serializing replication stats by @klauspost in https://github.com/minio/minio/pull/16007
    • Merge versions across sets when listing by @klauspost in https://github.com/minio/minio/pull/16003
    • force all internal MinIO operations to be under UTC by @harshavardhana in https://github.com/minio/minio/pull/16009
    • add deadlineConnections on remoteTransport by @harshavardhana in https://github.com/minio/minio/pull/16010
    • skip flaky tests on windows OS by @harshavardhana in https://github.com/minio/minio/pull/16015
    • allow changing argument for path for SNSD setup by @harshavardhana in https://github.com/minio/minio/pull/16013
    • initialize streaming events without lazy initialization by @harshavardhana in https://github.com/minio/minio/pull/16016
    • Update version of madmin-go to v1.7.3 by @anjalshireesh in https://github.com/minio/minio/pull/15994
    • Filter rest errors and logs returned by @klauspost in https://github.com/minio/minio/pull/16019
    • Add LDAP policy entities API by @donatello in https://github.com/minio/minio/pull/15908

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-10-29T06-21-33Z...RELEASE.2022-11-08T05-27-07Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-10-29T06-21-33Z(Oct 29, 2022)

    Highlights

    • First release to support rebalancing two or more pooled setups to even out disk usage refer #15483
    • Removes remaining gateway implementations of NAS/S3 and legacy FS mode completely - refer #15929. Existing users should continue to use their current releases if they wish to continue using gateway mode. Subscribed customers get downstream support until they are subscribed as per the terms of the subscription.
    • Other miscellaneous fixes in Site Replication, Multipart Checksums and SUBNET registration.

    What's Changed

    • Filter out tiering metadata during CopyObject by @krisis in https://github.com/minio/minio/pull/15936
    • remove gateway completely by @harshavardhana in https://github.com/minio/minio/pull/15929
    • Disallow configuring replication config when site replication is enabled by @poornas in https://github.com/minio/minio/pull/15937
    • site replication: proxy multipart to peer by @poornas in https://github.com/minio/minio/pull/15926
    • feat: Pool Rebalance by @krisis in https://github.com/minio/minio/pull/15483
    • site replication: replicate MakeBucket to peers in background by @poornas in https://github.com/minio/minio/pull/15945
    • Return better error status code in the lock handler by @vadmeste in https://github.com/minio/minio/pull/15950
    • Fix lock name to disallow parallel same erasure set healing by @vadmeste in https://github.com/minio/minio/pull/15951
    • add cross-build linux/amd64 by @yanggangtony in https://github.com/minio/minio/pull/15949
    • Ensure ilm expiry creates non null deletemarker for versioned bucket by @poornas in https://github.com/minio/minio/pull/15947
    • Store multipart checksums by @klauspost in https://github.com/minio/minio/pull/15953
    • make subnet subsys dynamic and simplify callhome by @harshavardhana in https://github.com/minio/minio/pull/15927
    • remove numAvailableDisks check as it doesn't serve any purpose by @harshavardhana in https://github.com/minio/minio/pull/15954
    • Fix inconsistent replication delete marker timestamps by @klauspost in https://github.com/minio/minio/pull/15956
    • xl-meta: update metadata version by @poornas in https://github.com/minio/minio/pull/15958
    • Mark subnet config keys as sensitive info by @anjalshireesh in https://github.com/minio/minio/pull/15962
    • Make pubsub type type safe by @klauspost in https://github.com/minio/minio/pull/15961
    • add a message of removal for gateway and hide the command by @harshavardhana in https://github.com/minio/minio/pull/15965
    • improve validation for replication resync API by @poornas in https://github.com/minio/minio/pull/15964

    New Contributors

    • @yanggangtony made their first contribution in https://github.com/minio/minio/pull/15949

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-10-24T18-35-07Z...RELEASE.2022-10-29T06-21-33Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-10-24T18-35-07Z(Oct 24, 2022)

    What's Changed

    • Fix Prometheus Probe tlsConfig, doesn't belong in ProberSpec by @briend in https://github.com/minio/minio/pull/15739
    • Audit dangling object removal by @vadmeste in https://github.com/minio/minio/pull/15933

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-10-21T22-37-48Z...RELEASE.2022-10-24T18-35-07Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-10-21T22-37-48Z(Oct 22, 2022)

    What's Changed

    • Do not freeze s3 traffic in healthinfo api by @anjalshireesh in https://github.com/minio/minio/pull/15912
    • lifecycle: Fix rules filtering by @vadmeste in https://github.com/minio/minio/pull/15914
    • Pass encrypted etag as is for immediate tiering by @krisis in https://github.com/minio/minio/pull/15925
    • Set CONSOLE_MINIO_SERVER to 127.0.0.1 by default by @vadmeste in https://github.com/minio/minio/pull/15887
    • Fix deletemarker replication check properly by @poornas in https://github.com/minio/minio/pull/15923
    • Check for s3zip content offset by @klauspost in https://github.com/minio/minio/pull/15924

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-10-20T00-55-09Z...RELEASE.2022-10-21T22-37-48Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-10-20T00-55-09Z(Oct 20, 2022)

    What's Changed

    • do not assume invalid buf to be non-xl.meta by @harshavardhana in https://github.com/minio/minio/pull/15843
    • convert context.DeadlineExceed to offline disk in DiskInfo() by @vadmeste in https://github.com/minio/minio/pull/15886
    • remove nancy, we rely on vulncheck from now on by @harshavardhana in https://github.com/minio/minio/pull/15893
    • do not need to trace ignored objects by @harshavardhana in https://github.com/minio/minio/pull/15894
    • Add concurrent Snowball extraction + options by @klauspost in https://github.com/minio/minio/pull/15836
    • storage: Return errDiskNotFound when a peer is during shutdown by @vadmeste in https://github.com/minio/minio/pull/15868
    • add filter support for tags and metadata in batch replication by @harshavardhana in https://github.com/minio/minio/pull/15885
    • Deprecate --listeners flag by @vadmeste in https://github.com/minio/minio/pull/15900
    • Add LDAP IDP Configuration APIs by @donatello in https://github.com/minio/minio/pull/15840
    • fix: AccountInfo API for roleARN based accounts by @donatello in https://github.com/minio/minio/pull/15907
    • fix: Delete() of bucket metadata should not parse the config by @harshavardhana in https://github.com/minio/minio/pull/15904

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-10-15T19-57-03Z...RELEASE.2022-10-20T00-55-09Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-10-15T19-57-03Z(Oct 16, 2022)

    What's Changed

    • lock tests: Initialize different DRWMutex for each lock by @vadmeste in https://github.com/minio/minio/pull/15833
    • Add missing server info fields by @klauspost in https://github.com/minio/minio/pull/15826
    • indicate how long it took to bring the drive online by @harshavardhana in https://github.com/minio/minio/pull/15835
    • honor specified target prefix under batch replication by @harshavardhana in https://github.com/minio/minio/pull/15834
    • remove spurious logging for object not found by @harshavardhana in https://github.com/minio/minio/pull/15842
    • Add metrics, version and apis handlers by @reivaj05 in https://github.com/minio/minio/pull/15839
    • fix: unrecognized openid config parameters by @hellivan in https://github.com/minio/minio/pull/15847
    • fix: replication stats() to not crash under any situation by @harshavardhana in https://github.com/minio/minio/pull/15851
    • Tweak objectQuorumFromMeta() to search for parity quorum by @vadmeste in https://github.com/minio/minio/pull/15844
    • do not disable IsKubernetes(), IsDocker() checks with MINIO_CI_CD by @harshavardhana in https://github.com/minio/minio/pull/15852
    • Omit empty checksums in responses by @klauspost in https://github.com/minio/minio/pull/15850
    • Bump golang.org/x/text to 0.3.8 to fix CVE-2022-32149 by @vadmeste in https://github.com/minio/minio/pull/15855
    • support deleteMarkers and all versions in batch replication by @harshavardhana in https://github.com/minio/minio/pull/15858
    • heal-info: Return the endpoint of a disk with unknown state by @vadmeste in https://github.com/minio/minio/pull/15854
    • validate setBucketTarget properly as per BucketExists() call by @harshavardhana in https://github.com/minio/minio/pull/15860
    • Convert context.DeadlineExceed to Offline disk in DiskInfo() by @vadmeste in https://github.com/minio/minio/pull/15869
    • validate bucket before attempting batch replication by @harshavardhana in https://github.com/minio/minio/pull/15861
    • support CRC32 Checksums on single drive setup by @harshavardhana in https://github.com/minio/minio/pull/15873
    • Pass role ARN for OIDC providers to console by @donatello in https://github.com/minio/minio/pull/15862

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-10-08T20-11-00Z...RELEASE.2022-10-15T19-57-03Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-10-08T20-11-00Z(Oct 9, 2022)

    Highlights

    • First release to include Batch Replication between MinIO to MinIO deployments - refer #15554

    What's Changed

    • set bucket creation timestamp properly for legacy FS backend by @harshavardhana in https://github.com/minio/minio/pull/15800
    • decom: Ignore object/version error during deletion by @vadmeste in https://github.com/minio/minio/pull/15806
    • remove build asset caching by @harshavardhana in https://github.com/minio/minio/pull/15807
    • Fix GetObjectRetention to parse in iso8601 time format by @poornas in https://github.com/minio/minio/pull/15809
    • Update pkg to v1.4.5 by @reivaj05 in https://github.com/minio/minio/pull/15808
    • remove unused debug param from evalActionFromLifecycle by @harshavardhana in https://github.com/minio/minio/pull/15813
    • Handle overlapping and conflicting ILM rules by @krisis in https://github.com/minio/minio/pull/15812
    • ignore VersionNotFound in addition to ObjectNotFound while replicating by @vadmeste in https://github.com/minio/minio/pull/15814
    • preserve Version and DeleteMarker sort order in the list XML response by @vadmeste in https://github.com/minio/minio/pull/15819
    • authenticate the request first for headObject() by @harshavardhana in https://github.com/minio/minio/pull/15820
    • Move ldap type to pkg to share with console by @donatello in https://github.com/minio/minio/pull/15810
    • marshal retention XML in expected format by @harshavardhana in https://github.com/minio/minio/pull/15821

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-10-05T14-58-27Z...RELEASE.2022-10-08T20-11-00Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-10-05T14-58-27Z(Oct 6, 2022)

    What's Changed

    • remove 'perf' tests from health diagnostics by @harshavardhana in https://github.com/minio/minio/pull/15780
    • Implement KMS handlers by @reivaj05 in https://github.com/minio/minio/pull/15737
    • site replication: fix typo in meta collection by @poornas in https://github.com/minio/minio/pull/15792
    • fix: heal service accounts for LDAP users in site replication by @harshavardhana in https://github.com/minio/minio/pull/15785
    • Use admin request check for ReplicationDiff handler by @vadmeste in https://github.com/minio/minio/pull/15793
    • Warn when 0 parity has been set as default parity by @klauspost in https://github.com/minio/minio/pull/15790
    • Removing unused getUpdateReaderFromFile function by @Alevsk in https://github.com/minio/minio/pull/15794

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-10-02T19-29-29Z...RELEASE.2022-10-05T14-58-27Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-10-02T19-29-29Z(Oct 3, 2022)

    What's Changed

    • Add RPC tcp timeout/errs and avg duration to prometheus by @vadmeste in https://github.com/minio/minio/pull/15747
    • fix: generalize SC config and also skip healing sub-sys under SD by @harshavardhana in https://github.com/minio/minio/pull/15757
    • Quick initialized of bucket target notifications by @vadmeste in https://github.com/minio/minio/pull/15571
    • support tagging based policy conditions by @harshavardhana in https://github.com/minio/minio/pull/15763
    • Update links to documentation site by @djwfyi in https://github.com/minio/minio/pull/15750
    • Fix Audit kafka configuration loading by @vadmeste in https://github.com/minio/minio/pull/15766
    • Fix race when accessing REST tcp dial values by @vadmeste in https://github.com/minio/minio/pull/15770
    • Re-load config after reset by @anjalshireesh in https://github.com/minio/minio/pull/15771
    • fix: de-duplicate conflicting object names on namespace by @harshavardhana in https://github.com/minio/minio/pull/15772
    • rename deprecated 'mc policy' -> 'mc anonymous' by @harshavardhana in https://github.com/minio/minio/pull/15779
    • allow tagging policy condition for GetObject by @harshavardhana in https://github.com/minio/minio/pull/15777

    New Contributors

    • @djwfyi made their first contribution in https://github.com/minio/minio/pull/15750

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-09-25T15-44-53Z...RELEASE.2022-10-02T19-29-29Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-09-25T15-44-53Z(Sep 26, 2022)

    What's Changed

    • fix: parse and save retainUntilDate in correct time format by @harshavardhana in https://github.com/minio/minio/pull/15741
    • validate correct ETag for the parts sent during CompleteMultipart by @harshavardhana in https://github.com/minio/minio/pull/15751
    • Relax retention date enforcement on replication target by @poornas in https://github.com/minio/minio/pull/15752
    • Add auto configuration of replication workers by @poornas in https://github.com/minio/minio/pull/15636
    • remove unused package internal/smart by @harshavardhana in https://github.com/minio/minio/pull/15758

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-09-22T18-57-27Z...RELEASE.2022-09-25T15-44-53Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-09-22T18-57-27Z(Sep 23, 2022)

    What's Changed

    • Helm conditions by @mersl in https://github.com/minio/minio/pull/15599
    • helm: default additional labels/annotations to object by @kasuboski in https://github.com/minio/minio/pull/15698
    • enable cross compile for openbsd/amd64 by @harshavardhana in https://github.com/minio/minio/pull/15701
    • Implement KMS methods for keys, policies and identities by @reivaj05 in https://github.com/minio/minio/pull/15673
    • Remove deprecated io/ioutil by @klauspost in https://github.com/minio/minio/pull/15707
    • helm: specify service account for the jobs by @cowjen01 in https://github.com/minio/minio/pull/15706
    • site replication: clear config if remove --all specified by @poornas in https://github.com/minio/minio/pull/15716
    • Fix cancellation leaks by @jiuker in https://github.com/minio/minio/pull/15721
    • persist the non-default creds in config by @harshavardhana in https://github.com/minio/minio/pull/15711
    • Fix ignoring odirect in case of erasure single disk by @vadmeste in https://github.com/minio/minio/pull/15734
    • lifecycle: Assign unique id to rules with empty id by @krisis in https://github.com/minio/minio/pull/15731
    • Freeze before exit when _MINIO_DEBUG_BLOCK_ON_FATAL_CRASH is defined by @vadmeste in https://github.com/minio/minio/pull/15709

    New Contributors

    • @mersl made their first contribution in https://github.com/minio/minio/pull/15599
    • @kasuboski made their first contribution in https://github.com/minio/minio/pull/15698
    • @reivaj05 made their first contribution in https://github.com/minio/minio/pull/15673

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-09-17T00-09-45Z...RELEASE.2022-09-22T18-57-27Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-09-17T00-09-45Z(Sep 17, 2022)

    What's Changed

    • limit number of buckets to 500k by @harshavardhana in https://github.com/minio/minio/pull/15668
    • Add StopDiskAtLimit by @klauspost in https://github.com/minio/minio/pull/15670
    • add vulncheck action to CI by @aead in https://github.com/minio/minio/pull/15676
    • Persist in-memory replication stats to disk by @poornas in https://github.com/minio/minio/pull/15594
    • prometheus: Track errors during REST read/write calls by @vadmeste in https://github.com/minio/minio/pull/15678
    • Fix S3 Select CSV -> JSON with variable field count by @klauspost in https://github.com/minio/minio/pull/15677
    • use go1.19.1 for vulncheck by @harshavardhana in https://github.com/minio/minio/pull/15680
    • Pass groups claim into condition values by @donatello in https://github.com/minio/minio/pull/15679
    • panic if replication config could not be read from disk by @poornas in https://github.com/minio/minio/pull/15685
    • update docs to add cycle in scanner by @sinhaashish in https://github.com/minio/minio/pull/15686
    • Fix flaky TestGetObjectWithOutdatedDisks by @klauspost in https://github.com/minio/minio/pull/15687
    • When listing, do not count delete markers by @klauspost in https://github.com/minio/minio/pull/15689
    • change replication stats save path by @poornas in https://github.com/minio/minio/pull/15690
    • add pre-conditions support for PUT calls during replication by @harshavardhana in https://github.com/minio/minio/pull/15674
    • optimize double reads by reusing results from checkUploadIDExists() by @harshavardhana in https://github.com/minio/minio/pull/15692
    • upgrade all deps and update CREDITS by @harshavardhana in https://github.com/minio/minio/pull/15650

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-09-07T22-25-02Z...RELEASE.2022-09-17T00-09-45Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-09-07T22-25-02Z(Sep 8, 2022)

    What's Changed

    • avoid printing io.ErrUnexpectedEOF for .metacache objects by @harshavardhana in https://github.com/minio/minio/pull/15642
    • do not print object not found errors in MRF healing by @harshavardhana in https://github.com/minio/minio/pull/15646
    • fis deadlock caused by unlocking a non-existing lock by @yudoutingle in https://github.com/minio/minio/pull/15635
    • fix: use optimized json.NewEncoder instead for metrics by @harshavardhana in https://github.com/minio/minio/pull/15648
    • Reuse madmin.ClusterRegistrationInfo by @anjalshireesh in https://github.com/minio/minio/pull/15654
    • verify RenameData() versions to be consistent by @harshavardhana in https://github.com/minio/minio/pull/15649
    • allow non-standards fallback for Expires header by @harshavardhana in https://github.com/minio/minio/pull/15655
    • Helm Chart: Use a Prometheus Operator Probe to scrape Cluster metrics by @briend in https://github.com/minio/minio/pull/15659
    • allow non-standards fallback for all http.TimeFormats by @harshavardhana in https://github.com/minio/minio/pull/15662
    • fix: trigger Heal when xl.meta needs healing during PUT by @harshavardhana in https://github.com/minio/minio/pull/15661
    • serialize updates to healing tracker by @krisis in https://github.com/minio/minio/pull/15647
    • Update PR template to make it easy to request for doc changes by @vadmeste in https://github.com/minio/minio/pull/15665

    New Contributors

    • @yudoutingle made their first contribution in https://github.com/minio/minio/pull/15635
    • @briend made their first contribution in https://github.com/minio/minio/pull/15659

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-09-01T23-53-36Z...RELEASE.2022-09-07T22-25-02Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-09-01T23-53-36Z(Sep 2, 2022)

    What's Changed

    • helm: add extra volumes and extra volume mounts option for job resources by @cowjen01 in https://github.com/minio/minio/pull/15590
    • improve performance for inlined data by @harshavardhana in https://github.com/minio/minio/pull/15603
    • site replication: Avoid returning root svcacct info in sr metadata by @poornas in https://github.com/minio/minio/pull/15608
    • Add extended checksum by @klauspost in https://github.com/minio/minio/pull/15433
    • audit-log: Add time to get object from remote-tier by @krisis in https://github.com/minio/minio/pull/15602
    • Avoid messages from go test output by @vadmeste in https://github.com/minio/minio/pull/15601
    • chore: remove duplicate word in comments by @Abirdcfly in https://github.com/minio/minio/pull/15607
    • feat: Add healing trace by @klauspost in https://github.com/minio/minio/pull/15616
    • Encrypt checksums by @klauspost in https://github.com/minio/minio/pull/15620
    • fix: add healing metric type for heal tracing by @harshavardhana in https://github.com/minio/minio/pull/15631
    • audit: account for response headers separately by @krisis in https://github.com/minio/minio/pull/15610
    • tier: avoid stats infinite loop in forwardTo method by @harshavardhana in https://github.com/minio/minio/pull/15640
    • Avoid replicating dirs in listing with replication enabled by @vadmeste in https://github.com/minio/minio/pull/15641
    • Save resync status in the backend using a last update timestamp by @vadmeste in https://github.com/minio/minio/pull/15638

    New Contributors

    • @Abirdcfly made their first contribution in https://github.com/minio/minio/pull/15607

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-26T19-53-15Z...RELEASE.2022-09-01T23-53-36Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-26T19-53-15Z(Aug 29, 2022)

    What's Changed

    • fix: validate deployment id when adding peer clusters by @poornas in https://github.com/minio/minio/pull/15591
    • Add concurrency to healing objects on a fresh disk by @krisis in https://github.com/minio/minio/pull/15575
    • fix key 'linters-settings' is duplicated by @jiuker in https://github.com/minio/minio/pull/15589
    • upgrade golang-lint to the latest by @harshavardhana in https://github.com/minio/minio/pull/15600
    • move adding port location so a pool which consist only one host got the port information by @FwP-IDN in https://github.com/minio/minio/pull/15585

    New Contributors

    • @FwP-IDN made their first contribution in https://github.com/minio/minio/pull/15585

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-25T07-17-05Z...RELEASE.2022-08-26T19-53-15Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-25T07-17-05Z(Aug 25, 2022)

    What's Changed

    • Properly replicate policy mapping for virtual users by @vadmeste in https://github.com/minio/minio/pull/15558
    • Supply proper initial values for annotations by @dingxiangfei2009 in https://github.com/minio/minio/pull/15570
    • helm: add extra volumes and extra volume mounts option by @cowjen01 in https://github.com/minio/minio/pull/15568
    • use new xxml for XML responses to support rare control characters by @harshavardhana in https://github.com/minio/minio/pull/15511
    • Factor out external event notification funcs by @donatello in https://github.com/minio/minio/pull/15574
    • Add listing block reuse by @klauspost in https://github.com/minio/minio/pull/15579
    • Fix regression in notifying peers about new policy mapping by @vadmeste in https://github.com/minio/minio/pull/15583
    • Ensure that only one node performs site replication healing by @vadmeste in https://github.com/minio/minio/pull/15584
    • fix: objects matching prefixes should not leave delete markers by @harshavardhana in https://github.com/minio/minio/pull/15586
    • Allow querying a single target in config get API by @donatello in https://github.com/minio/minio/pull/15587

    New Contributors

    • @dingxiangfei2009 made their first contribution in https://github.com/minio/minio/pull/15570

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-22T23-53-06Z...RELEASE.2022-08-25T07-17-05Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-22T23-53-06Z(Aug 23, 2022)

    Highlights

    • Versioning now supports Idempotent delete markers #15521
    • Disk IO statistics is now available as real-time monitoring via mc #15512
    • Replication now more accurately tracks failed replication events #15556

    What's Changed

    • helm: fix correct deployment environment variable by @sebtiz13 in https://github.com/minio/minio/pull/15534
    • Introduce disk io stats metrics by @vadmeste in https://github.com/minio/minio/pull/15512
    • fix: trim arn:aws:kms from incoming SSE aws-kms-key-id by @harshavardhana in https://github.com/minio/minio/pull/15540
    • replication: centralize healthcheck for remote targets by @poornas in https://github.com/minio/minio/pull/15516
    • Ensure subnet+callhome configs are single sub-sys by @donatello in https://github.com/minio/minio/pull/15542
    • mount export volume even if persistence is disabled by @rootmout in https://github.com/minio/minio/pull/15543
    • fix: increase concurrency of DeleteObjects() to N/10th by @harshavardhana in https://github.com/minio/minio/pull/15546
    • avoid using errors.As for 'errors.New' use errors.Is by @harshavardhana in https://github.com/minio/minio/pull/15549
    • Fixes after moving config parsing code in madmin by @donatello in https://github.com/minio/minio/pull/15552
    • feat: add idempotent delete marker support by @harshavardhana in https://github.com/minio/minio/pull/15521
    • fix: walk() should cancel itself upon context cancellation by @harshavardhana in https://github.com/minio/minio/pull/15553
    • typo in parameter name by @agowa338 in https://github.com/minio/minio/pull/15555
    • change default lock retry interval to 50ms by @harshavardhana in https://github.com/minio/minio/pull/15560
    • Fix error types in internal config package by @donatello in https://github.com/minio/minio/pull/15559
    • fix: ensure metadata update happens after deletemarker replication by @poornas in https://github.com/minio/minio/pull/15564
    • replication: improve MRF healing. by @poornas in https://github.com/minio/minio/pull/15556

    New Contributors

    • @sebtiz13 made their first contribution in https://github.com/minio/minio/pull/15534
    • @rootmout made their first contribution in https://github.com/minio/minio/pull/15543
    • @agowa338 made their first contribution in https://github.com/minio/minio/pull/15555

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-13T21-54-44Z...RELEASE.2022-08-22T23-53-06Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-13T21-54-44Z(Aug 14, 2022)

    What's Changed

    • Fix env var output in config get/export APIs by @donatello in https://github.com/minio/minio/pull/15528
    • init console after IAM init() by @harshavardhana in https://github.com/minio/minio/pull/15531
    • Move some config constants to madmin by @donatello in https://github.com/minio/minio/pull/15529
    • helm: (Bugfix) enable objectlocking by @reyerdam in https://github.com/minio/minio/pull/15524

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-11T04-37-28Z...RELEASE.2022-08-13T21-54-44Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-11T04-37-28Z(Aug 12, 2022)

    What's Changed

    • fix: maxConcurrent '0' is an invalid value by @harshavardhana in https://github.com/minio/minio/pull/15500
    • LDAP/OpenID must be initialized IAM Init() by @harshavardhana in https://github.com/minio/minio/pull/15491
    • restrict number of disks used for scanning buckets upto GOMAXPROCS by @harshavardhana in https://github.com/minio/minio/pull/15492
    • replication: heal proactively upon access by @poornas in https://github.com/minio/minio/pull/15501
    • site replication: fix group updates to set status correctly. by @poornas in https://github.com/minio/minio/pull/15507
    • fix: incorrect context timeout during listPath() by @harshavardhana in https://github.com/minio/minio/pull/15509
    • Upgrade madmin-go to latest version (v1.4.15) by @anjalshireesh in https://github.com/minio/minio/pull/15510
    • helm: Support adding objectlocking for buckets by @reyerdam in https://github.com/minio/minio/pull/15505
    • chore: bad title syntax of helm chart README by @noahziheng in https://github.com/minio/minio/pull/15513
    • Add trace support for decommissioning by @krisis in https://github.com/minio/minio/pull/15502

    New Contributors

    • @reyerdam made their first contribution in https://github.com/minio/minio/pull/15505
    • @noahziheng made their first contribution in https://github.com/minio/minio/pull/15513

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-08T18-34-09Z...RELEASE.2022-08-11T04-37-28Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-08T18-34-09Z(Aug 8, 2022)

    What's Changed

    • remove IAM old migration code by @harshavardhana in https://github.com/minio/minio/pull/15476
    • helm fix: add a new line to the end of the credentials file when creating a user by @cowjen01 in https://github.com/minio/minio/pull/15485
    • fix: keep writing previous speedtest result by @harshavardhana in https://github.com/minio/minio/pull/15484
    • site replication: handle invalid endpoint errors by @poornas in https://github.com/minio/minio/pull/15499
    • remove non-working GetRawData() from FS mode by @harshavardhana in https://github.com/minio/minio/pull/15498

    New Contributors

    • @cowjen01 made their first contribution in https://github.com/minio/minio/pull/15485

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-05T23-27-09Z...RELEASE.2022-08-08T18-34-09Z

    Source code(tar.gz)
    Source code(zip)
  • RELEASE.2022-08-05T23-27-09Z(Aug 6, 2022)

    Highlights

    • mc admin config export now supports relevant configured environment variables, refer #15465
    • Object speedtest now supports latency measurements, refer #15479
    • Fixes a rare goroutine leak in IAM subsystem causing CPU spike, refer #15471

    What's Changed

    • Add support converting non-inlined to inlined by @harshavardhana in https://github.com/minio/minio/pull/15444
    • fix: log if there is readDir() failure with ListBuckets by @harshavardhana in https://github.com/minio/minio/pull/15461
    • Add OIDC to the HelmChart by @ktaf in https://github.com/minio/minio/pull/15469
    • Properly cancel listAllIAMConfigItems to avoid goroutine leak by @vadmeste in https://github.com/minio/minio/pull/15471
    • Replaces 'disk'=>'drive' visible to end user by @ebozduman in https://github.com/minio/minio/pull/15464
    • fix: decodeDirObject() in single drive DeleteObjects() call by @harshavardhana in https://github.com/minio/minio/pull/15477
    • Add env variables to configuration APIs output by @donatello in https://github.com/minio/minio/pull/15465
    • bucket metadata import: return all errors by @poornas in https://github.com/minio/minio/pull/15462
    • feat: add SpeedTest ResponseTimes and TTFB by @harshavardhana in https://github.com/minio/minio/pull/15479
    • Ensure lock/versioning permissions when creating a bucket by @vadmeste in https://github.com/minio/minio/pull/15432

    Full Changelog: https://github.com/minio/minio/compare/RELEASE.2022-08-02T23-59-16Z...RELEASE.2022-08-05T23-27-09Z

    Source code(tar.gz)
    Source code(zip)
Owner
High Performance, Kubernetes Native Object Storage
High Performance, Kubernetes Native Object Storage
GeeseFS is a high-performance, POSIX-ish S3 (Yandex, Amazon) file system written in Go

GeeseFS is a high-performance, POSIX-ish S3 (Yandex, Amazon) file system written in Go Overview GeeseFS allows you to mount an S3 bucket as a file sys

Yandex.Cloud 329 Jan 1, 2023
Go object files

Go object files C developers will be familiar with compiling C source files (.c) into object files (.o) before linking them into their final form. Did

Andrew Kutz 0 Feb 1, 2022
Abstract File Storage

afs - abstract file storage Please refer to CHANGELOG.md if you encounter breaking changes. Motivation Introduction Usage Matchers Content modifiers S

Viant, Inc 221 Dec 30, 2022
A PDF document generator with high level support for text, drawing and images

GoFPDF document generator Package go-pdf/fpdf implements a PDF document generator with high level support for text, drawing and images. Features UTF-8

null 166 Jan 4, 2023
High Performance, Kubernetes Native Object Storage

MinIO Quickstart Guide MinIO is a High Performance Object Storage released under Apache License v2.0. It is API compatible with Amazon S3 cloud storag

High Performance, Kubernetes Native Object Storage 36.9k Jan 2, 2023
High Performance, Kubernetes Native Object Storage

MinIO Quickstart Guide MinIO is a High Performance Object Storage released under GNU Affero General Public License v3.0. It is API compatible with Ama

Multi-Cloud Object Storage 36.9k Jan 2, 2023
A High Performance Object Storage released under Apache License

MinIO Quickstart Guide MinIO is a High Performance Object Storage released under Apache License v2.0. It is API compatible with Amazon S3 cloud storag

null 1 Sep 30, 2021
go-fastdfs 是一个简单的分布式文件系统(私有云存储),具有无中心、高性能,高可靠,免维护等优点,支持断点续传,分块上传,小文件合并,自动同步,自动修复。Go-fastdfs is a simple distributed file system (private cloud storage), with no center, high performance, high reliability, maintenance free and other advantages, support breakpoint continuation, block upload, small file merge, automatic synchronization, automatic repair.(similar fastdfs).

中文 English 愿景:为用户提供最简单、可靠、高效的分布式文件系统。 go-fastdfs是一个基于http协议的分布式文件系统,它基于大道至简的设计理念,一切从简设计,使得它的运维及扩展变得更加简单,它具有高性能、高可靠、无中心、免维护等优点。 大家担心的是这么简单的文件系统,靠不靠谱,可不

小张 3.3k Jan 8, 2023
🔑A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout (LSM+WAL) similar to Riak.

bitcask A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout

James Mills 10 Sep 26, 2022
the pluto is a gateway new time, high performance, high stable, high availability, easy to use

pluto the pluto is a gateway new time, high performance, high stable, high availability, easy to use Acknowledgments thanks nbio for providing low lev

mobus 2 Sep 19, 2021
QingStor Object Storage service support for go-storage

go-services-qingstor QingStor Object Storage service support for go-storage. Install go get github.com/minhjh/go-service-qingstor/v3 Usage import ( "

minhjh 0 Dec 13, 2021
topolvm operator provide kubernetes local storage which is light weight and high performance

Topolvm-Operator Topolvm-Operator is an open source cloud-native local storage orchestrator for Kubernetes, which bases on topolvm. Supported environm

Alauda.io 24 Nov 24, 2022
An high performance and ops-free local storage solution for Kubernetes.

Carina carina 是一个CSI插件,在Kubernetes集群中提供本地存储持久卷 项目状态:开发测试中 CSI Version: 1.3.0 Carina architecture 支持的环境 Kubernetes:1.20 1.19 1.18 Node OS:Linux Filesys

BoCloud 10 May 18, 2022
Carina: an high performance and ops-free local storage for kubernetes

Carina English | 中文 Background Storage systems are complex! There are more and more kubernetes native storage systems nowadays and stateful applicatio

null 557 Dec 30, 2022
High Performance Remote Object Service Engine

=============== Hprose is a High Performance Remote Object Service Engine. It is a modern, lightweight, cross-language, cross-platform, object-oriente

小马哥 615 Nov 3, 2022
The Container Storage Interface (CSI) Driver for Fortress Block Storage This driver allows you to use Fortress Block Storage with your container orchestrator

fortress-csi The Container Storage Interface (CSI) Driver for Fortress Block Storage This driver allows you to use Fortress Block Storage with your co

Fortress 0 Jan 23, 2022
High-Performance server for NATS, the cloud native messaging system.

NATS is a simple, secure and performant communications system for digital systems, services and devices. NATS is part of the Cloud Native Computing Fo

NATS - The Cloud Native Messaging System 12k Jan 8, 2023
High-Performance server for NATS, the cloud native messaging system.

NATS is a simple, secure and performant communications system for digital systems, services and devices. NATS is part of the Cloud Native Computing Fo

NATS - The Cloud Native Messaging System 12k Jan 2, 2023
Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package.

Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package. The library allows you to call Go service methods from PHP with a minimal footprint, structures and []byte support.

Spiral Scout 1.1k Dec 28, 2022