A Kubernetes Native Batch System (Project under CNCF)

Related tags

Machine Learning kubernetes golang machine-learning gene hpc bigdata batch-systems
Overview

volcano-logo

Build Status Go Report Card RepoSize Release LICENSE CII Best Practices

Volcano is a batch system built on Kubernetes. It provides a suite of mechanisms that are commonly required by many classes of batch & elastic workload including: machine learning/deep learning, bioinformatics/genomics and other "big data" applications. These types of applications typically run on generalized domain frameworks like TensorFlow, Spark, PyTorch, MPI, etc, which Volcano integrates with.

Volcano builds upon a decade and a half of experience running a wide variety of high performance workloads at scale using several systems and platforms, combined with best-of-breed ideas and practices from the open source community.

NOTE: the scheduler is built based on kube-batch; refer to #241 and #288 for more detail.

cncf_logo

Volcano is a sandbox project of the Cloud Native Computing Foundation (CNCF). Please consider joining the CNCF if you are an organization that wants to take an active role in supporting the growth and evolution of the cloud native ecosystem.

Overall Architecture

volcano

Talks

Ecosystem

Quick Start Guide

Prerequisites

  • Kubernetes 1.12+ with CRD support

You can try Volcano by one of the following two ways.

Note:

  • For Kubernetes v1.16+ use CRDs under config/crd/bases (recommended)
  • For Kubernetes versions < v1.16 use CRDs under config/crd/v1beta1 (deprecated)

Install with YAML files

Install Volcano on an existing Kubernetes cluster. This way is both available for x86_64 and arm64 architecture.

For x86_64:
kubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/master/installer/volcano-development.yaml

For arm64:
kubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/master/installer/volcano-development-arm64.yaml

Enjoy! Volcano will create the following resources in volcano-system namespace.

NAME                                       READY   STATUS      RESTARTS   AGE
pod/volcano-admission-5bd5756f79-dnr4l     1/1     Running     0          96s
pod/volcano-admission-init-4hjpx           0/1     Completed   0          96s
pod/volcano-controllers-687948d9c8-nw4b4   1/1     Running     0          96s
pod/volcano-scheduler-94998fc64-4z8kh      1/1     Running     0          96s

NAME                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/volcano-admission-service   ClusterIP   10.98.152.108   <none>        443/TCP   96s

NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/volcano-admission     1/1     1            1           96s
deployment.apps/volcano-controllers   1/1     1            1           96s
deployment.apps/volcano-scheduler     1/1     1            1           96s

NAME                                             DESIRED   CURRENT   READY   AGE
replicaset.apps/volcano-admission-5bd5756f79     1         1         1       96s
replicaset.apps/volcano-controllers-687948d9c8   1         1         1       96s
replicaset.apps/volcano-scheduler-94998fc64      1         1         1       96s

NAME                               COMPLETIONS   DURATION   AGE
job.batch/volcano-admission-init   1/1           48s        96s

Install from code

If you don't have a kubernetes cluster, try one-click install from code base:

./hack/local-up-volcano.sh

This way is only available for x86_64 temporarily.

Install monitoring system

If you want to get prometheus and grafana volcano dashboard after volcano installed, try following commands:

make TAG=latest generate-yaml
kubectl create -f _output/release/volcano-monitoring-latest.yaml

Meeting

Regular Community Meeting:

The Volcano team meets once per week on Friday, alternating between 10am Beijing Time (Convert to your timezone.) and 3pm Beijing Time (Convert to your timezone.)

Resources:

Contact

If you have any question, feel free to reach out to us in the following ways:

CNCF Slack Channel

Mailing List

Comments
  • Failed to launch mpijob after installing volcano

    Failed to launch mpijob after installing volcano

    Hi everyone, I am trying to use the gang-scheduler in my k8s/kubeflow cluster and installed volcano following the tutorial here and here.

    $ kubectl get all -n volcano-system 
NAME                                       READY   STATUS      RESTARTS   AGE
pod/volcano-admission-5bd5756f79-5rxkh     1/1     Running     0          24h
pod/volcano-admission-init-nf2mc           0/1     Completed   0          24h
pod/volcano-controllers-687948d9c8-xclv7   1/1     Running     0          24h
pod/volcano-scheduler-79f569766f-bxgnf     1/1     Running     0          24h


NAME                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/volcano-admission-service   ClusterIP   10.107.67.206   <none>        443/TCP   24h


NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/volcano-admission     1/1     1            1           24h
deployment.apps/volcano-controllers   1/1     1            1           24h
deployment.apps/volcano-scheduler     1/1     1            1           24h

NAME                                             DESIRED   CURRENT   READY   AGE
replicaset.apps/volcano-admission-5bd5756f79     1         1         1       24h
replicaset.apps/volcano-controllers-687948d9c8   1         1         1       24h
replicaset.apps/volcano-scheduler-79f569766f     1         1         1       24h



NAME                               COMPLETIONS   DURATION   AGE
job.batch/volcano-admission-init   1/1           24s        24h

    However, some error messages came up when I launched the mpijob. It seems the job queue is not working properly.

    $ kubectl logs -n volcano-system volcano-controllers-687948d9c8-xclv7 --tail 10                                                                                             
I0917 02:26:57.418937       1 queue_controller.go:158] Begin sync queue default
I0917 02:26:57.418960       1 queue_controller.go:133] Error syncing queues "default", retrying. Error: queue default has not been seen or deleted
I0917 02:43:37.419076       1 queue_controller.go:158] Begin sync queue default
I0917 02:43:37.419106       1 queue_controller.go:133] Error syncing queues "default", retrying. Error: queue default has not been seen or deleted
I0917 03:00:17.419234       1 queue_controller.go:158] Begin sync queue default
I0917 03:00:17.419268       1 queue_controller.go:133] Error syncing queues "default", retrying. Error: queue default has not been seen or deleted
I0917 03:16:57.419408       1 queue_controller.go:158] Begin sync queue default
I0917 03:16:57.419431       1 queue_controller.go:133] Error syncing queues "default", retrying. Error: queue default has not been seen or deleted
I0917 03:33:37.419563       1 queue_controller.go:158] Begin sync queue default
I0917 03:33:37.419590       1 queue_controller.go:133] Error syncing queues "default", retrying. Error: queue default has not been seen or deleted

    The pods are all in "Pending" state

    $ kubectl get pods                 
NAME                                      READY   STATUS    RESTARTS   AGE
mxnet-horovod-job-launcher-7pncv          0/1     Pending   0          159m
mxnet-horovod-job-worker-0                0/1     Pending   0          159m
mxnet-horovod-job-worker-1                0/1     Pending   0          159m
mxnet-horovod-job-worker-2                0/1     Pending   0          159m
mxnet-horovod-job-worker-3                0/1     Pending   0          159m

    The output of the volcano-scheduler is like below

    $ kubectl logs -n volcano-system volcano-scheduler-79f569766f-bxgnf --tail 20
I0917 03:38:21.543470       1 enqueue.go:75] Try to enqueue PodGroup to 0 Queues
I0917 03:38:21.543496       1 enqueue.go:122] Leaving Enqueue ...
I0917 03:38:21.543509       1 allocate.go:43] Enter Allocate ...
I0917 03:38:21.543523       1 allocate.go:94] Try to allocate resource to 0 Namespaces
I0917 03:38:21.543544       1 allocate.go:247] Leaving Allocate ...
I0917 03:38:21.543552       1 backfill.go:42] Enter Backfill ...
I0917 03:38:21.543562       1 backfill.go:91] Leaving Backfill ...
I0917 03:38:21.547705       1 session.go:154] Close Session 989f0526-d8fc-11e9-af2b-46b0d5a5c4cd
I0917 03:38:22.548180       1 cache.go:771] There are <1> Jobs, <1> Queues and <7> Nodes in total for scheduling.
I0917 03:38:22.548205       1 session.go:135] Open Session 99386113-d8fc-11e9-af2b-46b0d5a5c4cd with <1> Job and <1> Queues
I0917 03:38:22.548540       1 enqueue.go:43] Enter Enqueue ...
I0917 03:38:22.548553       1 enqueue.go:58] Added Queue <default> for Job <default/mxnet-horovod-job>
I0917 03:38:22.548564       1 enqueue.go:75] Try to enqueue PodGroup to 0 Queues
I0917 03:38:22.548593       1 enqueue.go:122] Leaving Enqueue ...
I0917 03:38:22.548606       1 allocate.go:43] Enter Allocate ...
I0917 03:38:22.548621       1 allocate.go:94] Try to allocate resource to 0 Namespaces
I0917 03:38:22.548642       1 allocate.go:247] Leaving Allocate ...
I0917 03:38:22.548651       1 backfill.go:42] Enter Backfill ...
I0917 03:38:22.548662       1 backfill.go:91] Leaving Backfill ...
I0917 03:38:22.552921       1 session.go:154] Close Session 99386113-d8fc-11e9-af2b-46b0d5a5c4cd

    Really appreciate if someone can offer some help!

    opened by nicklhy 41
  • Large memory is used by volcano-scheduler

    Large memory is used by volcano-scheduler

    What happened: The volcano-scheduler uses large memory and the function of schedule seems to be unusual. The pod of job will always be pending. But there is no any error logs in pod. After restart it, everything is okay.

    The image link of the result of "kubectl top pods -n volcano-system": https://l4x826wg3c.feishu.cn/file/boxcnLnrbgq6CmvjAQWfOlWX9gc

    What you expected to happen: The schduler works well. Or know how to check what happened, then I could restart it when monitor some events.

    How to reproduce it (as minimally and precisely as possible): Not sure how to reproduce it, but it happend many times after running some days.

    Anything else we need to know?:

    1. The ETCD in our cluster uses normal Disk, not SSD.
    2. The Node volcano pods are deployed could be scheduled by other computer task like pytorchjob, and there is no Requests to volcano.

    Environment:

    • Volcano Version: https://github.com/volcano-sh/volcano/commit/1b96bdf4de821e1e4af2b4c056f67be7559a880d
    • Kubernetes version (use kubectl version): 1.22.2
    • Cloud provider or hardware configuration:
    • OS (e.g. from /etc/os-release):
    • Kernel (e.g. uname -a):
    • Install tools:
    • Others:
    kind/bug area/scheduling lifecycle/stale priority/important-soon 
    opened by jazka 38
  • Add GPU Numbers Predicates

    Add GPU Numbers Predicates

    Support specify GPU numbers for pod resource requests issue#1440

    Currently, Volcano only supports specified GPU share memory. Specified GPU number is not supported. This pr supports defining GPU numbers for pod resource requests. You can check the design doc https://github.com/peiniliu/volcano/blob/dev/docs/user-guide/how_to_use_gpu_number.md for more details.

    approved lgtm size/L priority/important-soon 
    opened by peiniliu 38
  • Switch to cross-compiled docker containers & container build

    Switch to cross-compiled docker containers & container build

    This allows for one container name to support multiple archs, and once pushed we can remove more of the arm64 specific installation stuff since the same container names will support both.

    docker does "the right thing" and pulls the container associated with the arch.

    You can take a look at the containers I built with this change in my own dockerhub at https://hub.docker.com/repository/docker/holdenk/volcanosh-scheduler , https://hub.docker.com/repository/docker/holdenk/volcanosh-controller-manager , https://hub.docker.com/repository/docker/holdenk/volcanosh-webhook-manager-base , etc.

    This is in response to https://github.com/volcano-sh/volcano/issues/1570 (although it could also solve https://github.com/volcano-sh/volcano/issues/1568 since we wouldn't need volcano-development-arm64.yaml anymore).

    To preserve backwards compatibility with users who might be developing locally in single arch mode I have that the default. If there are release docs I should update as well let me know.

    Signed-off-by: Holden Karau [email protected]

    approved lgtm ok-to-test size/L lifecycle/stale 
    opened by holdenk 35
  • Distinguish different pod-delete scenario

    Distinguish different pod-delete scenario

    Try to address issue #791 It's a draft solution, need further discussion.

    In my ENV, seems it could work, but the pg status not correct, after delete(after success) the original pod will gone and not recreate but the status of pg is:

    status:
  phase: Running
  running: 2

    not what expect to:

    status:
  phase: Running
  running: 2
  success: 1
    approved size/M lifecycle/stale 
    opened by vincent-pli 34
  • plugin ssh and mpi for HPC calculation for engine on earthquake

    plugin ssh and mpi for HPC calculation for engine on earthquake

    /kind feature

    Environment:

    • Volcano Version: 1.12
    • Kubernetes version (use kubectl version): Kind installation for testing: kubectl version Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.1", GitCommit:"c4d752765b3bbac2237bf87cf0b1c2e307844666", GitTreeState:"clean", BuildDate:"2020-12-18T12:09:25Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.4", GitCommit:"d360454c9bcd1634cf4cc52d1867af5491dc9c5f", GitTreeState:"clean", BuildDate:"2020-11-18T09:04:15Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}

    I want to use volcano as scheduler for our engine calculator for earthquakes. The communication of the cluster engine when we use VM or baremetal hosts is made by ssh

    I see that there are mpi plugin and also ssh plugin, but unfortunately I can't find any docs on what use these plugins in a deployment yaml. What i need is to understand in which way that plugin works to communicate from master to worker, look the follow example:

    apiVersion: batch.volcano.sh/v1alpha1
kind: Job
metadata:
  name: lm-mpi-job
spec:
  minAvailable: 3
  schedulerName: volcano
  plugins:
    ssh: []
    svc: []
  tasks:
    - replicas: 1
      name: mpimaster
      policies:
        - event: TaskCompleted
          action: CompleteJob
      template:
        spec:
          containers:
            - command:
                - /bin/sh
                - -c
                - |
                  sleep 10;
                  cat /etc/volcano/mpiworker.host | tr "\n" ","
                  MPI_HOST=`cat /etc/volcano/mpiworker.host | tr "\n" ","`;
                  mkdir -p /var/run/sshd; /usr/sbin/sshd;
                  mpiexec --allow-run-as-root --host ${MPI_HOST} -np 2 mpi_hello_world ;
                  sleep 100;
              image: volcanosh/example-mpi:0.0.1
              name: mpimaster
              ports:
                - containerPort: 22
                  name: mpijob-port
              workingDir: /home
          restartPolicy: OnFailure
    - replicas: 2
      name: mpiworker
      template:
        spec:
          containers:
            - command:
                - /bin/sh
                - -c
                - |
                  mkdir -p /var/run/sshd; /usr/sbin/sshd -D;
              image: volcanosh/example-mpi:0.0.1
              name: mpiworker
              ports:
                - containerPort: 22
                  name: mpijob-port
              workingDir: /home
          restartPolicy: OnFailure

    In this example the user is root, but is ti possible to use a different user for ssh plugin to can ssh to worker from master? Because on our image container we don't use user root but we need ssh connection from master to worker like open mpi And mpi plugin works in the same way? I find only a PR but no documentation on site volcano.sh or github available

    Thanks

    lifecycle/stale 
    opened by vot4anto 32
  • dynamically set tasks' replicas, with the range of [min, max]

    dynamically set tasks' replicas, with the range of [min, max]

    Is this a BUG REPORT or FEATURE REQUEST?:

    Uncomment only one, leave it on its own line:

    /kind bug kind feature

    What happened: Suppose in Tensorflow area, when user submits a distributed Tensorflow job, he must decide the number of workers, and usually set MinAvalible := sum of replicas of tasks.

    What you expected to happen: If we can set replicas to a range number, e.g. [min, max], we can enhance our scheduling ability.

    1. If there are enough resources, we can set as many workers as possible ( <= max)
    2. If there are less resources, we can start the job as fast as possible( >= min).
    3. If tensorflow workload (or any other workload) allow dynamic workers(e.g. auto-scaling), it gives kube-volcano more possibility to schedule.
    kind/feature area/scheduling lifecycle/stale priority/important-soon 
    opened by umialpha 30
  • SLA plugin doesn't work on `batch/v1` `Job` objects; `sla-waiting-time` from `volcano-scheduler.conf` is ignored

    SLA plugin doesn't work on `batch/v1` `Job` objects; `sla-waiting-time` from `volcano-scheduler.conf` is ignored

    What happened:

    As mentioned in #1869 I am using Volcano to schedule Kubernetes Job objects, to try and prevent smaller jobs submitted later from immediately filling any available space and starving larger jobs submitted earlier.

    My cluster has a 96-core node with hostname "k1.kube".

    I installed Volcano from the Helm chart in tag v1.4.0, using this values.yaml:

    basic:
  image_tag_version: "v1.4.0"
  controller_image_name: "volcanosh/vc-controller-manager"
  scheduler_image_name: "volcanosh/vc-scheduler"
  admission_image_name: "volcanosh/vc-webhook-manager"
  admission_secret_name: "volcano-admission-secret"
  admission_config_file: "config/volcano-admission.conf"
  scheduler_config_file: "config/volcano-scheduler.conf"

  image_pull_secret: ""
  admission_port: 8443
  crd_version: "v1"
custom:
  metrics_enable: "false"

    And then overriding the scheduler configmap with this and restarting the scheduler pod:

    apiVersion: v1
data:
  volcano-scheduler.conf: |
    actions: "enqueue, allocate, backfill"
    tiers:
    - plugins:
      - name: priority
      - name: gang
      - name: conformance
      - name: sla
        arguments:
          # Stop letting little jobs pass big jobs after the big jobs have been
          # waiting this long
          sla-waiting-time: 5m
    - plugins:
      - name: overcommit
      - name: drf
      - name: predicates
      - name: proportion
      - name: nodeorder
        arguments:
          # Maybe this will try to fill already full nodes first?
          leastrequested.weight: 0
          mostrequested.weight: 2
          nodeaffinity.weight: 3
          podaffinity.weight: 3
          balancedresource.weight: 1
          tainttoleration.weight: 1
          imagelocality.weight: 1
      - name: binpack
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: volcano
    meta.helm.sh/release-namespace: volcano-system
  labels:
    app.kubernetes.io/managed-by: Helm
  name: volcano-scheduler-configmap
  namespace: volcano-system

    So I should be using a global SLA of 5 minutes.

    Then, I prepared a test: fill up the node with some jobs, then queue a big job, then queue a bunch of smaller jobs after it:

    # Clean up
kubectl delete job -l app=volcanotest

# Make 10 10 core jobs that will block out our test job for at least 2 minutes
# Make sure they don't all finish at once.
rm -f jobs_before.yml
for NUM in {1..10} ; do
cat >>jobs_before.yml <<EOF
apiVersion: batch/v1
kind: Job
metadata:
  name: presleep${NUM}
  labels:
    app: volcanotest
spec:
  template:
    spec:
      schedulerName: volcano
      nodeSelector:
        kubernetes.io/hostname: k1.kube
      containers:
      - name: main
        image: ubuntu:20.04
        command: ["sleep",  "$(( $RANDOM % 20 + 120 ))"]
        resources:
          limits:
            memory: 300M
            cpu: 10000m
            ephemeral-storage: 1G
          requests:
            memory: 300M
            cpu: 10000m
            ephemeral-storage: 1G
      restartPolicy: Never
  backoffLimit: 4
  ttlSecondsAfterFinished: 1000
---
EOF
done

# And 200 10 core jobs that, if they all pass it, will keep it blocked out for 20 minutes
# We expect it really to be blocked like 5-7-10 minutes if the SLA plugin is working.
rm -f jobs_after.yml
for NUM in {1..200} ; do
cat >>jobs_after.yml <<EOF
apiVersion: batch/v1
kind: Job
metadata:
  name: postsleep${NUM}
  labels:
    app: volcanotest
spec:
  template:
    spec:
      schedulerName: volcano
      nodeSelector:
        kubernetes.io/hostname: k1.kube
      containers:
      - name: main
        image: ubuntu:20.04
        command: ["sleep",  "$(( $RANDOM % 20 + 60 ))"]
        resources:
          limits:
            memory: 300M
            cpu: 10000m
            ephemeral-storage: 1G
          requests:
            memory: 300M
            cpu: 10000m
            ephemeral-storage: 1G
      restartPolicy: Never
  backoffLimit: 4
  ttlSecondsAfterFinished: 1000
---
EOF
done

# And the test job itself between them.
rm -f job_middle.yml
cat >job_middle.yml <<EOF
apiVersion: batch/v1
kind: Job
metadata:
  name: middle
  labels:
    app: volcanotest
spec:
  template:
    spec:
      schedulerName: volcano
      nodeSelector:
        kubernetes.io/hostname: k1.kube
      containers:
      - name: main
        image: ubuntu:20.04
        command: ["sleep", "1"]
        resources:
          limits:
            memory: 300M
            cpu: 50000m
            ephemeral-storage: 1G
          requests:
            memory: 300M
            cpu: 50000m
            ephemeral-storage: 1G
      restartPolicy: Never
  backoffLimit: 4
  ttlSecondsAfterFinished: 1000
EOF

kubectl apply -f jobs_before.yml
sleep 10
kubectl apply -f job_middle.yml
sleep 10
CREATION_TIME="$(kubectl get job middle -o jsonpath='{.metadata.creationTimestamp}')"
kubectl apply -f jobs_after.yml
# Wait for it to finish
COMPLETION_TIME=""
while [[ -z "${COMPLETION_TIME}" ]] ; do
    sleep 10
    COMPLETION_TIME="$(kubectl get job middle -o jsonpath='{.status.completionTime}')"
done
echo "Test large job was created at ${CREATION_TIME} and completed at ${COMPLETION_TIME}"

    I observed jobs from jobs_after.yml being scheduled even when the job from job_middle.yml had had its pod pending for 10 minutes, which is double the global SLA time that should be being enforced.

    What you expected to happen:

    These shouldn't be much more than 5 minutes between the creation and completion times for the large middle job. When the job pod from job_middle.yml has been pending for 5 minutes, no more job pods from jobs_after.yml should be being scheduled by Volcano until job_middle.yml has been scheduled.

    How to reproduce it (as minimally and precisely as possible): Use the Volcano helm chart, the above configmap override, kubectl -n volcano-system delete pod "$(kubectl get pod -n volcano-system | grep volcano-scheduler | cut -f1 -d' ')" to bounce the schedule pod after reconfiguring it, and the above Bash code to generate test jobs. Adjust the hostname label selectors and job sizes as needed to fill the test cluster node you are using.

    Anything else we need to know?:

    Is the SLA plugin maybe not smart enough to clear out space for a job to meet the SLA from a node that matches its selectors? Are other plugins in the config maybe scheduling stuff that the SLA plugin has decided chouldn't be scheduled yet?

    The scheduler pod logs don't seem to include the string "sla", but they log a bunch for every pod that's waiting every second, so I might not be able to see the startup logs or every single line ever logged.

    The jobs are definitely getting PodGroups created for them. Here's the PodGroup description for the middle job when it should have been run according to the SLA but has not yet been:

    Name:         podgroup-31600c19-2282-47f1-934b-94026d88db1e
Namespace:    vg
Labels:       <none>
Annotations:  <none>
API Version:  scheduling.volcano.sh/v1beta1
Kind:         PodGroup
Metadata:
  Creation Timestamp:  2021-12-13T22:06:25Z
  Generation:          2
  Managed Fields:
    API Version:  scheduling.volcano.sh/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:ownerReferences:
      f:spec:
        .:
        f:minMember:
        f:minResources:
          .:
          f:cpu:
          f:ephemeral-storage:
          f:memory:
        f:priorityClassName:
      f:status:
    Manager:      vc-controller-manager
    Operation:    Update
    Time:         2021-12-13T22:06:25Z
    API Version:  scheduling.volcano.sh/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        f:conditions:
        f:phase:
    Manager:    vc-scheduler
    Operation:  Update
    Time:       2021-12-13T22:06:26Z
  Owner References:
    API Version:           batch/v1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Job
    Name:                  middle
    UID:                   31600c19-2282-47f1-934b-94026d88db1e
  Resource Version:        122332555
  Self Link:               /apis/scheduling.volcano.sh/v1beta1/namespaces/vg/podgroups/podgroup-31600c19-2282-47f1-934b-94026d88db1e
  UID:                     8bee9cca-40d5-47b5-90e7-ebb1bc70059a
Spec:
  Min Member:  1
  Min Resources:
    Cpu:                  50
    Ephemeral - Storage:  1G
    Memory:               300M
  Priority Class Name:    medium-priority
  Queue:                  default
Status:
  Conditions:
    Last Transition Time:  2021-12-13T22:06:26Z
    Message:               1/1 tasks in gang unschedulable: pod group is not ready, 1 Pending, 1 minAvailable; Pending: 1 Unschedulable
    Reason:                NotEnoughResources
    Status:                True
    Transition ID:         86f1b151-92dd-4893-bcd3-c2573b3029fc
    Type:                  Unschedulable
  Phase:                   Inqueue
Events:
  Type     Reason         Age                   From     Message
  ----     ------         ----                  ----     -------
  Warning  Unschedulable  64s (x1174 over 21m)  volcano  1/1 tasks in gang unschedulable: pod group is not ready, 1 Pending, 1 minAvailable; Pending: 1 Unschedulable

    Environment:

    • Volcano Version: v1.4.0
    • Kubernetes version (use kubectl version):
    Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.0", GitCommit:"e19964183377d0ec2052d1f1fa930c4d7575bd50", GitTreeState:"clean", BuildDate:"2020-08-26T14:23:04Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
    • Cloud provider or hardware configuration: Nodes are hosted on AWS instances.
    • OS (e.g. from /etc/os-release):
    NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
    • Kernel (e.g. uname -a):
    Linux master.kube 5.8.7-1.el7.elrepo.x86_64 #1 SMP Fri Sep 4 13:11:18 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux
    • Install tools:
    helm version
version.BuildInfo{Version:"v3.7.2", GitCommit:"663a896f4a815053445eec4153677ddc24a0a361", GitTreeState:"clean", GoVersion:"go1.16.10"}
    • Others:
    kind/bug lifecycle/stale 
    opened by adamnovak 29
  • Pass conformance test

    Pass conformance test

    Is this a BUG REPORT or FEATURE REQUEST?:

    /kind feature

    Description:

    Cherry pick related PR in kube-batch to volcano-sh/kube-batch for conformance test.

    /cc @asifdxtreme

    kind/feature priority/high 
    opened by k82cn 27
  • add admitPod and PGController

    add admitPod and PGController

    Which issue(s) this PR fixes : Fixes #135 #134

    Special notes for your reviewer:

    1. new func AdmitPod in admission controller

    2. new PGcontroller in controller

    3. delete Inqueue job phase

    4. fix UT

    Release note:

    
1. add ValidatingWebhookConfiguration volcano-validate-pod, only limit CREATE pods, allow pods to create when:
- pod.spec.schedulerName is default-scheduler
- podgroup phase isn't Pending
- normal job, no podgroup

2. new PGcontroller, create pg for normal job when use kube-batch.

3. if create job, job phase will be Pending->Running...... , so fix UT
    lgtm size/XXL needs-rebase 
    opened by wangyuqing4 26
  • Fair sharing not working

    Fair sharing not working

    What happened: My cluster has total 11 CPU. I'm trying to create 2 queue(excluding default queue) with weight 5 for each queue. Queue manifest,

    apiVersion: scheduling.volcano.sh/v1beta1
kind: Queue
metadata:
  name: test
spec:
  weight: 5

---

apiVersion: scheduling.volcano.sh/v1beta1
kind: Queue
metadata:
  name: test1
spec:
  weight: 5

    Queue List,

    Name                     Weight  State   Inqueue Pending Running Unknown
default                  1       Open    0       0       0       0
test                     5       Open    0       0       0       0
test1                    5       Open    0       0       0       0

    Created 3 Jobs for test queue with CPU resource as follow, job1 -> CPU 5 job2 -> CPU 5 job3 -> CPU 1

    Now all 3 jobs are running and utilizing full cluster.

    Now i'm creating new Job in test1 queue with CPU 2. I'm expecting 1 Job will be evicted from test queue and Job in test1 queue will be running. But Job in test1 queue is in Inqueue state.

    Name                     Weight  State   Inqueue Pending Running Unknown
default                  1       Open    0       0       0       0
test                     5       Open    0       0       3       0
test1                    5       Open    1       0       0       0

    Configuration,

    actions: "enqueue, allocate, backfill"
tiers:
- plugins:
  - name: priority
  - name: gang
  - name: conformance
- plugins:
  - name: drf
  - name: predicates
  - name: proportion
  - name: nodeorder
  - name: binpack

    What you expected to happen: I'm expecting 1 Job will be evicted from test queue and Job in test1 queue will be running. But Job in test1 queue is in Inqueue state. How to reproduce it (as minimally and precisely as possible):

    Anything else we need to know?:

    Environment:

    • Volcano Version: v1.3.0
    • Kubernetes version (use kubectl version):
    • Cloud provider or hardware configuration:
    • OS (e.g. from /etc/os-release):
    • Kernel (e.g. uname -a):
    • Install tools:
    • Others:
    kind/bug lifecycle/stale 
    opened by Sharathmk99 25
  • Error occur when execute the same task many times

    Error occur when execute the same task many times

    What happened: When test the performance of spark-native integration with volcano, when execute the same task with 50 times, interval is 3 seconds, half task failed.

    What you expected to happen: All task execute one by one and all of them should success.

    How to reproduce it (as minimally and precisely as possible): Executing task by the following code.

    #!/bin/bash
s=0
for ((i=1;i<=50;i=i+1))
do
     nohup ${SPARK_HOME1}/bin/spark-submit \
     --master k8s://https://10.32.226.132:6443 \
     --deploy-mode cluster \
     --class cn.cestc.test.JavaSparkReadHiveInHahadoopForComponent \
     --driver-cores 1 \
     --driver-memory 2G \
     --num-executors 1 \
     --executor-cores 1 \
     --executor-memory 2G \
     --name native_modetask2 \
     --jars hdfs://dev-host-03:8082/dolphinscheduler_arm/supportarm/resources/performance1/mysql-connector-java-8.0.29.jar \
     --conf spark.executor.instances=1 \
     --conf spark.kubernetes.namespace=support132x86 \
     --conf spark.kubernetes.authenticate.driver.serviceAccountName=support132x86 \
     --conf spark.kubernetes.container.image=10.32.226.224:85/public-release/spark-volcano:3.3.0  \
     --conf spark.kubernetes.scheduler.name=volcano  \
     --conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/tmp/podgroup-template.yaml \
     --conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep \
     --conf spark.kubernetes.executor.pod.featureSteps=org.apache.spark.deploy.k8s.features.VolcanoFeatureStep \
  hdfs://dev-host-03:8082/dolphinscheduler_x86/support132x86/resources/performance1/SparkJupiterTest-2.0.jar\
 datacluster hdfs://datacluster/user/hive/warehouse \
thrift://dev-host-01:9083,thrift://dev-host-02:9083,thrift://dev-host-03:9083\
 dev-host-01:8082 dev-host-03:8082 testdata\
 hdfs://datacluster/dolphinscheduler_arm/supportarm/resources/performance1/yuanj2.sql \
10.32.226.69 30306 testdata root CESTC1Dhr7El67KD3jG@ out_test_task2 2000 > log-$i.log &
     sleep 3s
done

    podgroup-template.yaml

    apiVersion: scheduling.volcano.sh/v1beta1
kind: PodGroup
spec:
  # Specify minMember to 1 to make a driver pod
  minMember: 1
  # Specify minResources to support resource reservation (the driver pod resource and executors pod resource should be considered)
  # It is useful for ensource the available resources meet the minimum requirements of the Spark job and avoiding the
  # situation where drivers are scheduled, and then they are unable to schedule sufficient executors to progress.
  minResources:
    cpu: "10"
    memory: "20G"
  # Specify the priority, help users to specify job priority in the queue during scheduling.
  priorityClassName: system-node-critical
  # Specify the queue, indicates the resource queue which the job should be submitted to
  queue: support132x86

    Anything else we need to know?: error task log

    [INFO] 2022-12-29 16:46:15.060 [TaskLogInfo- - [taskAppId=TASK-7947514526080_14-1674-6086]-getOutputLogService]  -  -> 22/12/29 16:46:14 ERROR Client: Please check "kubectl auth can-i create pod" first. It should be yes.
	Exception in thread "main" io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://10.32.226.132:6443/api/v1/namespaces/support132x86/pods. Message: admission webhook "validatepod.volcano.sh" denied the request: failed to create pod <support132x86/spkj-6086-driver> as the podgroup phase is Pending. Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=admission webhook "validatepod.volcano.sh" denied the request: failed to create pod <support132x86/spkj-6086-driver> as the podgroup phase is Pending, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=null, status=Failure, additionalProperties={}).
		at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:682)
		at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:661)
		at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:612)
		at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:555)
		at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:518)
		at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:305)
		at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:644)
		at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:83)
		at io.fabric8.kubernetes.client.dsl.base.CreateOnlyResourceOperation.create(CreateOnlyResourceOperation.java:61)
		at org.apache.spark.deploy.k8s.submit.Client.run(KubernetesClientApplication.scala:152)
		at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$5(KubernetesClientApplication.scala:248)
		at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$5$adapted(KubernetesClientApplication.scala:242)
		at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:2764)
		at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.run(KubernetesClientApplication.scala:242)
		at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.start(KubernetesClientApplication.scala:214)
		at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958)
		at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
		at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
		at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
		at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046)
		at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055)
		at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
	22/12/29 16:46:14 INFO ShutdownHookManager: Shutdown hook called
	22/12/29 16:46:15 INFO ShutdownHookManager: Deleting directory /tmp/spark-8da822e9-7f18-433a-886e-eb8b56529792
[INFO] 2022-12-29 16:46:16.152 [TaskLogInfo- - [taskAppId=TASK-7947514526080_14-1674-6086]-getOutputLogService]  - FINALIZE_SESSION
[INFO] 2022-12-29 16:46:16.162 [TaskLogInfo- - [taskAppId=TASK-7947514526080_14-1674-6086]]  - process has exited, execute path:/tmp/dolphinschedu

    Environment:

    • Volcano Version: latest
    • Kubernetes version (use kubectl version):
    Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.10", GitCommit:"98d5dc5d36d34a7ee13368a7893dcb400ec4e566", GitTreeState:"clean", BuildDate:"2021-04-15T03:28:42Z", GoVersion:"go1.15.10", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.10", GitCommit:"98d5dc5d36d34a7ee13368a7893dcb400ec4e566", GitTreeState:"clean", BuildDate:"2021-04-15T03:20:25Z", GoVersion:"go1.15.10", Compiler:"gc", Platform:"linux/amd64"}
    • Cloud provider or hardware configuration:
    • OS (e.g. from /etc/os-release):
    • Kernel (e.g. uname -a):Linux master 5.4.121-1.el7.elrepo.x86_64 #1 SMP Thu May 20 19:22:37 EDT 2021 x86_64 x86_64 x86_64 GNU/Linux
    • Install tools:
    • Others:
    kind/bug 
    opened by a092cc 1
  • remove gox

    remove gox

    Signed-off-by: hwdef [email protected]

    Removing the gox tool has the following benefits:

    • Simplify project construction complexity
    • Reduce the packages that the project depends on
    • Compilation is smoother in China's network environment, because it is likely to fail when installing gox
    size/M 
    opened by hwdef 1
  • Preempt do not follow minAvailable.

    Preempt do not follow minAvailable.

    What happened:

    When I tried preempt action with priority plugin. I set minAvailable: 1 in high-priority job which has 4 tasks, and supposed that Volcano would evict only 1 task of low-priority job. But the result was that all 4 tasks from high-priority job started and evicted 4 low-priority tasks as below:

    NAME               READY   STATUS        RESTARTS   AGE
job-high-task0-0   0/1     Pending       0          3s
job-high-task0-1   0/1     Pending       0          3s
job-high-task1-0   0/1     Pending       0          3s
job-high-task1-1   0/1     Pending       0          3s
job-low-task0-0    1/1     Terminating   0          13m
job-low-task0-1    1/1     Terminating   0          13m
job-low-task1-0    1/1     Terminating   0          13m
job-low-task1-1    1/1     Terminating   0          14m

    What you expected to happen:

    How to reproduce it (as minimally and precisely as possible):

    volcano-scheduler-configmap was set as below:

        actions: "enqueue, allocate, preempt, backfill"
    tiers:
    - plugins:
      - name: priority
      - name: gang
        enableJobStarving: false
        enablePreemptable: false
      - name: conformance
    - plugins:
      - name: drf
        enablePreemptable: false
      - name: predicates
      - name: nodeorder
      - name: binpack
    1. Set priorityclass:
    NAME                      VALUE        GLOBAL-DEFAULT   AGE
high                      1000         false            49d
low                       10           false            31h
    1. Apply low-priority job with 4 task and fullfilled the cluster:
    apiVersion: batch.volcano.sh/v1alpha1
kind: Job
metadata:
  name: job-low
spec:
  schedulerName: volcano
  minAvailable: 4
  queue: default
  priorityClassName: low
  tasks:
    - replicas: 2
      name: "task0"
      template:
        spec:
          containers:
            - image: alpine
              command: ["/bin/sh", "-c", "sleep 99999"]
              imagePullPolicy: IfNotPresent
              name: task
              resources:
                limits:
                  cpu: 3750m
                requests:
                  cpu: 3750m
          restartPolicy: OnFailure
    - replicas: 2
      name: "task1"
      template:
        spec:
          containers:
            - image: alpine
              command: ["/bin/sh", "-c", "sleep 99999"]
              imagePullPolicy: IfNotPresent
              name: task
              resources:
                limits:
                  cpu: 3750m
                requests:
                  cpu: 3750m
          restartPolicy: OnFailure
    1. Apply high-priority job with 4 task but minAvailable: 1:
    apiVersion: batch.volcano.sh/v1alpha1
kind: Job
metadata:
  name: job-high
spec:
  schedulerName: volcano
  minAvailable: 1
  queue: default
  priorityClassName: high
  tasks:
    - replicas: 2
      name: "task0"
      template:
        spec:
          containers:
            - image: alpine
              command: ["/bin/sh", "-c", "sleep 99999"]
              imagePullPolicy: IfNotPresent
              name: task
              resources:
                limits:
                  cpu: 3750m
                requests:
                  cpu: 3750m
          restartPolicy: OnFailure
    - replicas: 2
      name: "task1"
      template:
        spec:
          containers:
            - image: alpine
              command: ["/bin/sh", "-c", "sleep 99999"]
              imagePullPolicy: IfNotPresent
              name: task
              resources:
                limits:
                  cpu: 3750m
                requests:
                  cpu: 3750m
          restartPolicy: OnFailure
    1. Wait for preempt.

    Anything else we need to know?:

    Environment:

    • Volcano Version: latest
    • Kubernetes version (use kubectl version): 1.21
    • Cloud provider or hardware configuration:
    • OS (e.g. from /etc/os-release):
    • Kernel (e.g. uname -a):
    • Install tools:
    • Others:
    kind/bug 
    opened by jiangkaihua 5
  • allocateIdleResource Method Calculate Node Idle Bugs

    allocateIdleResource Method Calculate Node Idle Bugs

    What happened: When using volcano to allocate some task into a node, there are UnexpectedAdmissionError events involved with the pod. The volcano scheduler will assign different pod to the same node because of the lack of gpu resources. As result, many job failed in a short period of time.

    Here was error message:

    Allocate failed due to requested number of devices unavailable for nvidia.com/gpu. Requested: 4, Available: 3, which is unexpected

    What you expected to happen: Node info in the vc-scheduler should be calculated precisely and correctly, so that the task will not be scheduled to the node.

    How to reproduce it (as minimally and precisely as possible): At some special moment, there was a node with 4 GPUs which are 3 healthy, 1 unhealthy, and used by an inference task. In the log of scheduler, the idle of the node was 3 GPUs and the used of the node was 4GPUs, causing the bugs.

    Anything else we need to know?:

    func (ni *NodeInfo) allocateIdleResource(ti *TaskInfo) error {
	if ti.Resreq.LessEqual(ni.Idle, Zero) {
		ni.Idle.Sub(ti.Resreq)
		return nil
	}

	return &AllocateFailError{Reason: fmt.Sprintf(
		"cannot allocate resource, <%s> idle: %s <%s/%s> req: %s",
		ni.Name, ni.Idle.String(), ti.Namespace, ti.Name, ti.Resreq.String(),
	)}
}

    In pkg/scheduler/api/node_info.go, when Resreq was larger than node idle, the node idle didn't set to the correct value.

    kind/bug 
    opened by RamezesDong 0
Releases(v1.6.0)

  • v1.6.0(Jun 11, 2022)

    What's New

    Support Dynamic Scheduling Based on Real Node Load

    This feature aims to schedule pods based on real node load instead of request resource, which will optimize the node resource utilization.Currently the pod is scheduled based on the request resources and node allocatable resources other than the node usage. This leads to the unbalanced resource usage of compute nodes. Pod is scheduled to node with higher usage and lower allocation rate. This is not what users expect. Users expect the usage of each node to be balanced. More details can be referred to https://github.com/volcano-sh/volcano/blob/master/docs/design/usage-based-scheduling.md. (https://github.com/volcano-sh/volcano/pull/2023, https://github.com/volcano-sh/volcano/pull/2129 @william-wang )

    Support Rescheduling Based on Real Node Load

    This feature enables users to rebalance the node utilization based on real node resource usage reqularlly, which is quite suitable for long-running workloads such as deployment. All the rescheduling policies and check interval can be configured according to custom scenarios. More details can be referred to https://github.com/volcano-sh/volcano/blob/master/docs/design/rescheduling.md. (https://github.com/volcano-sh/volcano/pull/2174, https://github.com/volcano-sh/volcano/pull/2184 @Thor-wl )

    Support Elastic Job Scheduling

    This feature allows Volcano to schedule volcano job based on the [min,max] configuration in the job, which will improve resource utilization rate and shorten the execution time of training job. More details can be referred to https://github.com/volcano-sh/volcano/blob/master/docs/design/elastic-scheduler.md. (https://github.com/volcano-sh/volcano/pull/2105, @qiankunli )

    Add MPI Job Plugin

    This feature provides a new volcano job plugin - MPI Plugin. It will be more convient for MPI users to make use of volcano job instead of manually making connections for hosts of different roles, registering required environment variables and so on. More details can be referred to https://github.com/volcano-sh/volcano/blob/master/docs/design/distributed-framework-plugins.md. (https://github.com/volcano-sh/volcano/pull/2237, @hwdef )

    Other Notable Changes

    • update helm version in install.sh(https://github.com/volcano-sh/volcano/pull/2103, @hwdef )
    • modify the way to install the controller-gen(https://github.com/volcano-sh/volcano/pull/2104, @hwdef )
    • add shuffle action(https://github.com/volcano-sh/volcano/pull/2174, @Thor-wl )
    • add e2e Spark integration test(https://github.com/volcano-sh/volcano/pull/2113, @Yikun )
    • if only one candidate node, no need do scoring for it(https://github.com/volcano-sh/volcano/pull/2122, @wpeng102 )
    • skip verify init container SecurityContex.Privileged(https://github.com/volcano-sh/volcano/pull/2125, @zrss )
    • add design doc for usage based scheduling(https://github.com/volcano-sh/volcano/pull/2023, @william-wang )
    • add usage based scheduling plugin(https://github.com/volcano-sh/volcano/pull/2129, @william-wang )
    • support elastic annotation in preempt/reclaim plugin(https://github.com/volcano-sh/volcano/pull/2105, @qiankunli )
    • add design doc for Enhance-Generate-PodGroup-OwnerReferences-for-Normal-Pod(https://github.com/volcano-sh/volcano/pull/2151, @wpeng102 )
    • allow no retry when task failed(https://github.com/volcano-sh/volcano/pull/2154, @merryzhou )
    • remove useless code in task-topology's manager.go(https://github.com/volcano-sh/volcano/pull/2159, @HeGaoYuan )
    • add user guidance for svc plugin(https://github.com/volcano-sh/volcano/pull/2162, @Thor-wl )
    • add user guidance of env plugin(https://github.com/volcano-sh/volcano/pull/2153, @Thor-wl )
    • add user guidance for ssh plugin(https://github.com/volcano-sh/volcano/pull/2168, @Thor-wl )
    • add user guidance about how to configure volcano scheduler(https://github.com/volcano-sh/volcano/pull/2177, @Thor-wl )
    • add user guidance about how to configure job and task policy(https://github.com/volcano-sh/volcano/pull/2179, @Thor-wl )
    • add overhead for pod request(https://github.com/volcano-sh/volcano/pull/2170, @jiangxiaobin96 )
    • rename ClusterRole from prometheus to prometheus-volcano(https://github.com/volcano-sh/volcano/pull/2178, @SimonYang-CS )
    • add image pull secret for volcano-admission-init job(https://github.com/volcano-sh/volcano/pull/2185, @SimonYang-CS )
    • add rescheduling plugin(https://github.com/volcano-sh/volcano/pull/2184, @Thor-wl )
    • feat(scheduler): support resource quota consideration during pod group enqueue procedure(https://github.com/volcano-sh/volcano/pull/1345, @merryzhou )
    • add priorityClassName for rescheduler(https://github.com/volcano-sh/volcano/pull/2200, @jiangxiaobin96 )
    • allow privilege containers to pass the admission webhook validation by default(https://github.com/volcano-sh/volcano/pull/2222, @Thor-wl )
    • clean up metrics of deleted objects(https://github.com/volcano-sh/volcano/pull/2230, @xiaoanyunfei )
    • sunset the reservation plugin and elect reserve actions(https://github.com/volcano-sh/volcano/pull/2236, @william-wang )
    • add more deploy switches on helm(https://github.com/volcano-sh/volcano/pull/2267, @shinytang6 )

    Bug Fixes

    • fix dynamic provision ut case error(https://github.com/volcano-sh/volcano/pull/2133, @wpeng102 )
    • fix: add jobUID into job's podgroup name ensure podgroup's unique(https://github.com/volcano-sh/volcano/pull/2140, @FengXingYuXin)
    • fix: Add mirror for Spark voclano IT(https://github.com/volcano-sh/volcano/pull/2163, @Yikun )
    • fix controller job cache not sync latest version issue(https://github.com/volcano-sh/volcano/pull/2169, @wpeng102 )
    • fix: add jobUID into job's podgroup name ensure podgroup's unique(https://github.com/volcano-sh/volcano/pull/2140, @FengXingYuXin )
    • fix task MinAvailable issue(https://github.com/volcano-sh/volcano/pull/2176, @merryzhou )
    • fix calculate inqueue resource bug in opensession(https://github.com/volcano-sh/volcano/pull/2214, @zbbkeepgoing )
    • fix id of gpu devices never delete when number gpu decrease(https://github.com/volcano-sh/volcano/pull/2215, @WingkaiHo)
    • fix numa divided by zero(https://github.com/volcano-sh/volcano/pull/2216, @elinx)
    • fix helm install(https://github.com/volcano-sh/volcano/pull/2218, @zirain )
    • fix api-server deny empty admission response with PatchType set(https://github.com/volcano-sh/volcano/pull/2267, @elinx)
    • feat exclude unhealthy devices(https://github.com/volcano-sh/volcano/pull/2267, @yongjiahe)
    • fix unhealthy gpu data struc array(https://github.com/volcano-sh/volcano/pull/2267, @yongjiahe)
    • fix high priority task cannot preemt low priority task when queue is overused(https://github.com/volcano-sh/volcano/pull/2267, @wpeng102 )
    • avoid panic for query prometheus no data(https://github.com/volcano-sh/volcano/pull/2267, @waiterQ )
    • modify prometheus.query.result judg(https://github.com/volcano-sh/volcano/pull/2267, @waiterQ )
    • fix(scheduler): fix jobStarvingFn logic(https://github.com/volcano-sh/volcano/pull/2271, @shinytang6 )
    Source code(tar.gz)
    Source code(zip)

  • v1.5.1(Mar 15, 2022)

    Changes since v1.5.0

    • bug fix: fix the driver pod can not be created due to unreasonable admit (#2081 @william-wang )
    • bug fix: fix error message in TestValidateJobCreate ( #2077 @william-wang )
    • bug fix: Open state queue can be deleted ( #2077 @Yikun )
    • bug fix: upgrade webhook from v1beta1 to v1 to make sure volcano webhook work on K8S 1.22+ ( #2077 @william-wang )
    • bug fix: fix the proportion plugin that ignore the inqueue resource in running jobs( #2057 @Thor-wl )
    • bug fix: set the initial phase to be pending for podgroup ( #2057 @Thor-wl )
    • bug fix: regenerate installer/volcano-development-arm64.yaml to fix arm64 deployment ( #2030 @hwdef )
    • bug fix: fix queue allocated exceeds capability ( #2035 @aidaizyy @Thor-wl )
    Source code(tar.gz)
    Source code(zip)

  • v1.5.0(Feb 20, 2022)

    Changes since v1.5.0-Beta

    • bug fix: fix some concurrent map bugs in numaware-aware(https://github.com/volcano-sh/volcano/pull/1968, @huone1 @Jason-Liu-Dream )
    • bug fix: fix the scheduler stuck after delete resourcequota for namespace(https://github.com/volcano-sh/volcano/pull/1978, @william-wang )
    • bug fix: add individual development yamls for volcano v1.5(https://github.com/volcano-sh/volcano/pull/2004, @hwdef )
    Source code(tar.gz)
    Source code(zip)

  • v1.4.1(Feb 20, 2022)

    Changes since v1.4.0

    • bug fix: fix panic in setNodeState function when node is nil(https://github.com/volcano-sh/volcano/pull/1970, @Thor-wl )
    • bug fix: fix possible panic when 'SetNode' is called(https://github.com/volcano-sh/volcano/pull/1952, @william-wang )
    • bug fix: fix some concurrent map bugs in numaware-aware(https://github.com/volcano-sh/volcano/pull/1969, @huone1 @Jason-Liu-Dream )
    • bug fix: all pods is existing when restart count exceed max retry(https://github.com/volcano-sh/volcano/pull/1997, @william-wang )
    • bug fix: add individual development yamls for volcano v1.4(https://github.com/volcano-sh/volcano/pull/2002, @hwdef )
    • bug fix: optimize resource comparision functions for performance(https://github.com/volcano-sh/volcano/pull/2026, @huone1 )
    Source code(tar.gz)
    Source code(zip)

  • v1.5.0-Beta(Dec 31, 2021)

    What's New

    Support Task Dependency

    In most mainstream computing platforms such as MPI and Tensorflow, different pods undertake different roles, for example, master/worker. It is necessary to start master or worker first due to the working principle for different platforms. This feature aims to provide the ability to make the start order correct. More details please refer to https://github.com/volcano-sh/volcano/blob/master/docs/design/task-launch-order-within-job.md. (https://github.com/volcano-sh/volcano/pull/1920, https://github.com/volcano-sh/volcano/pull/1833, @hwdef @shinytang6 @Thor-wl )

    Support Reserve Resource for Queue

    This feature provides the ability to reserve resources for specified queues in order to make sure there is always guaranteed resources for urgent jobs instead of waiting for resource release or being preempted. More details please refer to https://github.com/volcano-sh/volcano/blob/master/docs/design/queue-guarantee-resource-reservation-design.md (https://github.com/volcano-sh/volcano/pull/1905, https://github.com/volcano-sh/volcano/pull/1904, @qiankunli )

    Support Specified Nodes for Volcano in Cluster

    In some scenarios such as multiple schedulers, it is necessary for Volcano to be only responsible for part of nodes in the cluster. This feature enable users to configure the nodes that are responsible for the Volcano. More details can be referred to https://github.com/volcano-sh/volcano/pull/1834 (https://github.com/volcano-sh/volcano/pull/1821, @qiankunli )

    Add Tendorflow Job Plugin

    Volcano provides a unified object for job management which allows user to run AI training such as Tensorflow, Pytorch, Mxnet, MPI with Volcano Job and enjoy the enhanced lifecycle management. However it is a bit complex for some users. This features is to add Tensorflow plugin based on Volcano job plugin framework which reduces the complexity of running Tensorflow with Volcano and make it easy to use. More details can be referred to https://github.com/volcano-sh/volcano/blob/master/docs/design/distributed-framework-plugins.md (https://github.com/volcano-sh/volcano/pull/1874, @LuBingtan )

    Other Notable Changes

    • update CRD version to v1(https://github.com/volcano-sh/volcano/pull/1919, @Thor-wl )
    • update golang to v1.17(https://github.com/volcano-sh/volcano/pull/1912, @Thor-wl )
    • optimize: reuse predicate error on same task group(https://github.com/volcano-sh/volcano/pull/1906, @justadogistaken )
    • default sort task by index(https://github.com/volcano-sh/volcano/pull/1898, @xiaoanyunfei )
    • update PriorityClass from v1beta1 to v1 for go-client(https://github.com/volcano-sh/volcano/pull/1897, @lc2705 )
    • support label volcano.sh/task-priority(https://github.com/volcano-sh/volcano/pull/1896, @qiankunli )
    • enhance the security of TLS client authentication for webhook(https://github.com/volcano-sh/volcano/pull/1895, @huone1 )
    • add healthz and metric switch for deploy controller and scheduler(https://github.com/volcano-sh/volcano/pull/1888, @huone1 )
    • add elastic scheduler design doc(https://github.com/volcano-sh/volcano/pull/1887, @qiankunli )
    • add eventhandler framework proposal(https://github.com/volcano-sh/volcano/pull/1886, @sivanzcw )
    • add a argument csi-storage to control the storage capacity resource(https://github.com/volcano-sh/volcano/pull/1875, @huone1 )
    • add rbac for csinode(https://github.com/volcano-sh/volcano/pull/1871, @Thor-wl )
    • feat: add imagelocality priority to nodeOrder(https://github.com/volcano-sh/volcano/pull/1868, @justadogistaken )
    • optimize the CA parse Process(https://github.com/volcano-sh/volcano/pull/1862, @huone1 )
    • ignore the update event if pod is allocated in cache but not present in NodeName(https://github.com/volcano-sh/volcano/pull/1857, @xing0821)
    • improve taintTolerationScore interPodAffinityScore throghput when failure occurs(https://github.com/volcano-sh/volcano/pull/1856, @justadogistaken )
    • switch the order of patch and existence check(https://github.com/volcano-sh/volcano/pull/1852, @zzr93)
    • support to set healthz address and metrics address(https://github.com/volcano-sh/volcano/pull/1849, @huone1 )
    • add nodeSelector design doc(https://github.com/volcano-sh/volcano/pull/1834, @qiankunli )
    • enhance the volcano topology framework(https://github.com/volcano-sh/volcano/pull/1762, @huone1 )
    • support preempt with priority plugin alone(https://github.com/volcano-sh/volcano/pull/1757, @Thor-wl )
    • support reserved node(https://github.com/volcano-sh/volcano/pull/1821, @qiankunli )
    • support multi-cluster scheduling in framework(https://github.com/volcano-sh/volcano/pull/1521, @william-wang )
    • cleanup scheduler cache informerFactory(https://github.com/volcano-sh/volcano/pull/1831, @xiaoanyunfei )
    • cleanup AddPriorityClass(https://github.com/volcano-sh/volcano/pull/1828, @xiaoanyunfei )
    • clean addNumaInfo(https://github.com/volcano-sh/volcano/pull/1829, @xiaoanyunfei )
    • clean up readAdmissionConf(https://github.com/volcano-sh/volcano/pull/1823, @xiaoanyunfei )
    • Remove default quota info in NewNamespaceCollection(https://github.com/volcano-sh/volcano/pull/1817, @zen-xu )
    • add multiple tasks support(https://github.com/volcano-sh/volcano/pull/1820, @hwdef )
    • refactor the cache to support batch bind api for better performance(https://github.com/volcano-sh/volcano/pull/1796, @huone1 )
    • optimize resource comparision functions for performance(https://github.com/volcano-sh/volcano/pull/1769, @huone1 )
    • optimize some logs in admission process(https://github.com/volcano-sh/volcano/pull/1738, @huone1 )
    • add setting MinResources to pg for normal pod(https://github.com/volcano-sh/volcano/pull/1666, @huone1 )
    • don't return err message when the pod isn't in the nodeinfo cache(https://github.com/volcano-sh/volcano/pull/1478, @huone1 )
    • update vendor for resource reservation(https://github.com/volcano-sh/volcano/pull/1494, @huone1 )
    • Proposal: Add Machine Learning Framework Plugins in Volcano(https://github.com/volcano-sh/volcano/pull/1806, @LuBingtan )
    • upgrade spf13/cobra version to 1.2.1(https://github.com/volcano-sh/volcano/pull/1801, @marffin)
    • refactor the volcano to support multi-scheduler with each job and node get conresponding scheduler based on hash.(https://github.com/volcano-sh/volcano/pull/1795, @william-wang )
    • support multi-scheduler for k8s workload deployment, etc(https://github.com/volcano-sh/volcano/pull/1792, @huone1 )
    • use root context(https://github.com/volcano-sh/volcano/pull/1715, @lowang-bh )
    • add design docs for task-leve advanced scheduling policy(https://github.com/volcano-sh/volcano/pull/1630, @hwdef )
    • Add livenessProbe and readinessProbe in Grafana Container(https://github.com/volcano-sh/volcano/pull/1788, @dipanjank )
    • enhance the admission conf check(https://github.com/volcano-sh/volcano/pull/1799, @huone1 )
    • Catch add pod out of sync error(https://github.com/volcano-sh/volcano/pull/1783, @zhiyuone )
    • add UT for elect action(https://github.com/volcano-sh/volcano/pull/1780, @Thor-wl )
    • Adding oidc import to enable vcctl work with oidc cluster(https://github.com/volcano-sh/volcano/pull/1793, @igormishsky)
    • Add job conditions (status&lastTransitionTime)(https://github.com/volcano-sh/volcano/pull/1764, @HecarimV )
    • add ut converage report for v1.4.0(https://github.com/volcano-sh/volcano/pull/1766, @Thor-wl )
    • refactor the Jobinfo functions to reduce redundant computing(https://github.com/volcano-sh/volcano/pull/1745, @william-wang )

    Bug Fixes

    • fix scheduling process starts even if resource synchronization is not complete(https://github.com/volcano-sh/volcano/pull/1916, @huone1 )
    • fix: allocate ut for "two Jobs on one node"(https://github.com/volcano-sh/volcano/pull/1913, @justadogistaken )
    • fix the deep clone of JobInfo(https://github.com/volcano-sh/volcano/pull/1883, @lc2705 )
    • fix the security alert from Kubernetes(https://github.com/volcano-sh/volcano/pull/1873, @Thor-wl )
    • fix pod cannot be allocated with sufficient resource(https://github.com/volcano-sh/volcano/pull/1851, @aidaizyy )
    • fix: avoid chan block within taintTolerationScore(https://github.com/volcano-sh/volcano/pull/1848, @justadogistaken )
    • fix: scheduler crash fatal error: concurrent map writes(https://github.com/volcano-sh/volcano/pull/1847, @Jason-Liu-Dream)
    • fix syntax error in function Remove GPUIndexPatch(https://github.com/volcano-sh/volcano/pull/1841, @Thor-wl )
    • fix: All pods is existing when restart count exceed max retry(https://github.com/volcano-sh/volcano/pull/1719, @LuBingtan )
    • fix there is nil pointer access in function setNodeState(https://github.com/volcano-sh/volcano/pull/1800, @huone1 )
    • fix OOM will occur if pod info is sync before node info(https://github.com/volcano-sh/volcano/pull/1662, @huone1 )
    • fix controller panic when create A large number of pods(https://github.com/volcano-sh/volcano/pull/1814, @huone1 )
    • fix a problem about equivalence ecache feature (https://github.com/volcano-sh/volcano/pull/1593, @huone1 )
    • fix(scheduler) gang plugin task min avaliable check(https://github.com/volcano-sh/volcano/pull/1732, @king-jingxiang)
    • fix: fix possible panic when 'SetNode' is called(https://github.com/volcano-sh/volcano/pull/1685, @eggiter )
    • fix bug that vcjob is not compeleted when maxRetry is 1(https://github.com/volcano-sh/volcano/pull/1746, @Thor-wl )
    • fix the security alerts(https://github.com/volcano-sh/volcano/pull/1770, @Thor-wl )
    • fix(scheduler): improve job/task clone func(https://github.com/volcano-sh/volcano/pull/1729, @shinytang6 )
    • fix broken grafana dashboard configuration.(https://github.com/volcano-sh/volcano/pull/1773, @dipanjank )
    Source code(tar.gz)
    Source code(zip)

  • v1.4.0(Sep 18, 2021)

    Changes since v1.4.0-Beta

    • fix bug about not record queue label in metric(https://github.com/volcano-sh/volcano/pull/1722, @lowang-bh )
    • fix: do not set taskInfo.NodeName to empty when nodeInfo.RemoveTask is called(https://github.com/volcano-sh/volcano/pull/1716, @eggiter )
    • fix(underused): Do not check overused when there is no UnderUsedResourceFn added(https://github.com/volcano-sh/volcano/pull/1726, @eggiter )
    • pass kubeClient to admission service(https://github.com/volcano-sh/volcano/pull/1730, @hack-qian)
    • upgrade k8s to v1.19.11 because of security notification(https://github.com/volcano-sh/volcano/pull/1733, @Thor-wl )
    • optimize some logs in admission process(https://github.com/volcano-sh/volcano/pull/1738, @huone1 )
    • change the Mutex to RWMutex in predicateCache(https://github.com/volcano-sh/volcano/pull/1741, @william-wang )
    • fix vcjob not work when mount volume(https://github.com/volcano-sh/volcano/pull/1742, @Thor-wl )
    • e2e cases about pod affinity skip cancel(https://github.com/volcano-sh/volcano/pull/1743, @Thor-wl )
    • fix bug that vcjob is not compeleted when maxRetry is 1(https://github.com/volcano-sh/volcano/pull/1746, @Thor-wl )
    • fix gen-admission-secret.sh(https://github.com/volcano-sh/volcano/pull/1752, @yahaa )
    Source code(tar.gz)
    Source code(zip)

  • v1.4.0-Beta(Sep 4, 2021)

    What's New

    1. Support multi-scheduler

    In Kubernetes cluster with multiple schedulers, different kinds of workloads should be mapped to certain scheduler sometimes. For example, K8s native workloads such as deployment in namespace kube-system are mapped to default-sheduler while AI and Big data jobs are mapped to Volcano. This feature aims to implements that automaticallty. More details please refer to https://github.com/volcano-sh/volcano/blob/master/docs/design/multi-scheduler.md. (https://github.com/volcano-sh/volcano/pull/1576, https://github.com/volcano-sh/volcano/pull/1521, @huone1 @william-wang )

    2. Support proportion of resources for GPU node

    In order to make full use of scarce resources such as GPU, one solution is to bind them with other resources as shares. For example, it is common to see a lot of CPU-intensive workloads are scheduled to GPU nodes. When GPU-intensive workloads come, they cannot be scheduled because of lack of CPU or Memory in GPU nodes. If workloads requiring both GPU, CPU, Memory at certatin range can be scheduled to GPU nodes first, it is possible to make full use of GPUs. More details please refer to https://github.com/volcano-sh/volcano/blob/master/docs/design/proportional.md. (https://github.com/volcano-sh/volcano/pull/1527, @king-jingxiang )

    3. Support CPU NUMA-Aware scheduling

    As to CPU-intensive workloads especially in AI, Big Data and HPC fileds, It will result in a significant performance improvement if CPU NUMA is enabled. More details please refer to https://github.com/volcano-sh/volcano/blob/master/docs/design/numa-aware.md. (https://github.com/volcano-sh/volcano/pull/1493, @huone1 )

    4. Provide framework of stress test

    In this release, A kind of framework for Volcano stress test is provided. (https://github.com/volcano-sh/volcano/pull/1516, @rudeigerc )

    Other Notable Changes

    • update roadmap for v1.5(https://github.com/volcano-sh/volcano/pull/1708, @william-wang )
    • refine scheduler framework(https://github.com/volcano-sh/volcano/pull/1705, @wpeng102 )
    • fix daily release job fails(https://github.com/volcano-sh/volcano/pull/1701, @Thor-wl )
    • optimization register controller log(https://github.com/volcano-sh/volcano/pull/1683, @hwdef )
    • lower ssh key access permission(https://github.com/volcano-sh/volcano/pull/1682, @hwdef )
    • automatically set GOMAXPROCS for scheduler & controller & webhook-manager(https://github.com/volcano-sh/volcano/pull/1681, @SataQiu )
    • add revive linter as drop-in replacement of archived golint(https://github.com/volcano-sh/volcano/pull/1678, @gy95 )
    • modify default scheduler cfg(https://github.com/volcano-sh/volcano/pull/1673, @shinytang6 )
    • expose detailed scheduling reason of pending tasks(https://github.com/volcano-sh/volcano/pull/1672, @eggiter)
    • format env key(https://github.com/volcano-sh/volcano/pull/1660, @wpeng102 )
    • not delete failed pods on last try(https://github.com/volcano-sh/volcano/pull/1657, @wpeng102 )
    • optimize function setOversubscription(https://github.com/volcano-sh/volcano/pull/1653, @huone1 )
    • upgrade controller tools to v0.6.0(https://github.com/volcano-sh/volcano/pull/1649, @shinytang6 )
    • enable configuring volcano webhooks(https://github.com/volcano-sh/volcano/pull/1645, @hacker-qian)
    • add user-provided rsa key-pair and mount rsa key-pair in InitContainers since we want to clone private git repository before run training task(https://github.com/volcano-sh/volcano/pull/1644, @python279 )
    • refactor the preempt function in plugin gang(https://github.com/volcano-sh/volcano/pull/1643, @huone1 )
    • delete bindingTasks from NodeInfo structure(https://github.com/volcano-sh/volcano/pull/1636, @huone1 )
    • scheduler plugin framework support SharedInformerFactory(https://github.com/volcano-sh/volcano/pull/1635, @wpeng102 )
    • add task-spec in pod label(https://github.com/volcano-sh/volcano/pull/1626, @python279 )
    • rename resource comparision functions(https://github.com/volcano-sh/volcano/pull/1624, @Thor-wl )
    • update resource comparision doc(https://github.com/volcano-sh/volcano/pull/1622, @Thor-wl )
    • add Equal function(https://github.com/volcano-sh/volcano/pull/1621, @Thor-wl )
    • add LessEqualPartly function(https://github.com/volcano-sh/volcano/pull/1613, @Thor-wl )
    • add LessEqualInAllDimension function and remove LessEqual/LessEqualStrict functions(https://github.com/volcano-sh/volcano/pull/1611, @Thor)
    • doc: add resource quota plugin doc(https://github.com/volcano-sh/volcano/pull/1583, @merryzhou )
    • improve: clean unready status when job ready(https://github.com/volcano-sh/volcano/pull/1582, @lowang-bh )
    • improve: reduce calculation of total nodes' resource in stead of storing it when snapshot(https://github.com/volcano-sh/volcano/pull/1578, @lowang-bh )
    • add resource comparison doc(https://github.com/volcano-sh/volcano/pull/1573, @Thor-wl )
    • update Less function(https://github.com/volcano-sh/volcano/pull/1569, @Thor-wl )
    • support oversubscription in volcano framework(https://github.com/volcano-sh/volcano/pull/1566, @wpeng102 )
    • add multi-scheduling design doc(https://github.com/volcano-sh/volcano/pull/1565, @huone1 )
    • improve: preempt break out if intersection is null(https://github.com/volcano-sh/volcano/pull/1563, @lowang-bh )
    • add arch node selectors to the installer arm&amd64(https://github.com/volcano-sh/volcano/pull/1556, @holdenk )
    • optimizing openSession(https://github.com/volcano-sh/volcano/pull/1541, @hacker-qian)
    • improve: remove nodename from taskinfo when RemoveTask(https://github.com/volcano-sh/volcano/pull/1517, @lowang-bh )
    • the member Others should be copied in func (ni *NodeInfo) Clone() (https://github.com/volcano-sh/volcano/pull/1512, @huone1 )
    • use struct NodeScoreList better than HostPriorityList for score(https://github.com/volcano-sh/volcano/pull/1510, @huone1 )
    • add min success design(https://github.com/volcano-sh/volcano/pull/1505, @zen-xu )
    • change min resource to 0.1 in resource_info(https://github.com/volcano-sh/volcano/pull/1489, @wpeng102 )

    Bug Fixes

    • fix diff function(https://github.com/volcano-sh/volcano/pull/1712, @Thor-wl )

    • make 'existing pods anti-affinity rules' work(https://github.com/volcano-sh/volcano/pull/1668, @eggiter)

    • add setting MinResources to pg for normal pod(https://github.com/volcano-sh/volcano/pull/1666, @huone1 )

    • fix OOM will occur if pod info is sync before node info(https://github.com/volcano-sh/volcano/pull/1662, @huone1 )

    • fix addmission parsing bug(https://github.com/volcano-sh/volcano/pull/1656, @hacker-qian)

    • fix overused judgement when deal with allocate and proportion(https://github.com/volcano-sh/volcano/pull/1637, @Thor-wl )

    • fix bugs about resource comparison(https://github.com/volcano-sh/volcano/pull/1628, @Thor-wl )

    • reset task.NodeName after call DeallocateFunc(https://github.com/volcano-sh/volcano/pull/1618, @merryzhou )

    • adds the missing field to the Webhook(https://github.com/volcano-sh/volcano/pull/1615, @hwdef )

    • fix a problem about equivalence ecache feature (https://github.com/volcano-sh/volcano/pull/1593, @huone1 )

    • func FeasibleNodesToFind to use list with a centain order(https://github.com/volcano-sh/volcano/pull/1574, @lowang-bh )

    • fix static check warning in pkg folder(https://github.com/volcano-sh/volcano/pull/1552, @hwdef )

    • fix bug in predicates plugin(https://github.com/volcano-sh/volcano/pull/1547, @hacker-qian)

    • fix(scheduler): reclaim action minus and comparison bug(https://github.com/volcano-sh/volcano/pull/1540, @shinytang6 )

    • fix resource comparasion bug in task topology(https://github.com/volcano-sh/volcano/pull/1546, @Thor-wl )

    • fix select wrong queue when proportion is disable(https://github.com/volcano-sh/volcano/pull/1497, @zen-xu )

    • fix the error for make verify(https://github.com/volcano-sh/volcano/pull/1508, @huone1 )

    Source code(tar.gz)
    Source code(zip)

  • v1.3.0(May 27, 2021)

    What's New

    1. Support minAvailable at task level

    Just as the minAvailable at job level, minAvailable at task level will regard replicases at the same task as group and decide whether to schedule pods at the task. Only when the minAvailable is meet will the pods will be scheduled together. More details please refer to https://github.com/volcano-sh/volcano/blob/master/docs/design/task-minavailable.md. (https://github.com/volcano-sh/volcano/pull/1459, @shinytang6 )

    2. Support minSuccess for Job

    Support to configure the least number of pods belonging to the job. It's useful to mark the status of job when minsuccess reaches or not and accelerates the job status judgement. (https://github.com/volcano-sh/volcano/pull/1384, @zen-xu )

    3. Support task-topology

    In big data processing jobs like Tensorflow & Spark, tasks transmitted a large amount of data between each other, causing transmission delay took a large proportion in job execution time. So task topology plugin was proposed to modify scheduling strategy according to transmission topology inside a job, so as to cut the data amount to be transmitted between nodes, decrease transmission delay proportion in job execution time, and improve resource utilization. More details please refer to https://github.com/volcano-sh/volcano/blob/master/docs/design/task-topology-plugin.md. (https://github.com/volcano-sh/volcano/pull/1353, @jiangkaihua )

    4. Create new repository volcano.sh/apis

    Separate apis from volcano.sh/volcanosh. Any downstream projects can introduce the CRD clientset/lister/informer with the K8s version it needs. (https://github.com/volcano-sh/apis, @Thor-wl )

    Other Notable Changes

    • fix the bug of CRD apiversion and installation volcano with kind(https://github.com/volcano-sh/volcano/pull/1483, @Thor-wl )
    • doc: add schedulerName in gpu sharing user guide(https://github.com/volcano-sh/volcano/pull/1481, @ChAnYaNG97 )
    • update scheduler default QPS and Burst(https://github.com/volcano-sh/volcano/pull/1480, @Thor-wl )
    • vcctl queue support get kubeconfig from env(https://github.com/volcano-sh/volcano/pull/1477, @yahaa )
    • add queue annotation in deployment example and add queue yaml(https://github.com/volcano-sh/volcano/pull/1474, @nolimitkun )
    • support K8s v1.19. (https://github.com/volcano-sh/volcano/pull/1444, @Thor-wl )
    • optimize yaml unmarshal logic(https://github.com/volcano-sh/volcano/pull/1427, @sniperking1234 )
    • doc: add multi schedulers design doc(https://github.com/volcano-sh/volcano/pull/1403, @zen-xu )
    • simplify unit-test(https://github.com/volcano-sh/volcano/pull/1394, @zen-xu )
    • add new target update-development-yaml in Makefile(https://github.com/volcano-sh/volcano/pull/1386, @zen-xu )
    • add additional printer columns to crd Job(https://github.com/volcano-sh/volcano/pull/1385, @zen-xu )
    • update ca.crt and server.csr validity period to 10 years(https://github.com/volcano-sh/volcano/pull/1382, @zen-xu )
    • Helm support crd v1(https://github.com/volcano-sh/volcano/pull/1378, @zen-xu )
    • feat(webhook): add podgroup admission(https://github.com/volcano-sh/volcano/pull/1375, @shinytang6 )
    • support auto updating crd manifests in helm templates when run make generate-yaml(https://github.com/volcano-sh/volcano/pull/1374, @zen-xu )
    • refactor(e2e): separate utils as a single package(https://github.com/volcano-sh/volcano/pull/1362, @rudeigerc )
    • support taint toleration preferNoScheduler in release-0.4(https://github.com/volcano-sh/volcano/pull/1354, @huone1 )
    • support taintToleration preferNoschdule(https://github.com/volcano-sh/volcano/pull/1352, @huone1 )

    Bug Fixes

    • fix: lose preemptor when considering Preemption between Tasks within same Job (https://github.com/volcano-sh/volcano/pull/1453, @lowang-bh )
    • scheduler need configmap role to enable elect funtion(https://github.com/volcano-sh/volcano/pull/1443, @wpeng102 )
    • fix(scheduler): use nodeMap to fix anti-affinity problem(https://github.com/volcano-sh/volcano/pull/1430. @shinytang6 )
    • fix: use task.Name to make podName in admission(https://github.com/volcano-sh/volcano/pull/1412, @merryzhou )
    • add bindingTasks to judge whether adding node to the snapshot.(https://github.com/volcano-sh/volcano/pull/1388, @zen-xu )
    • fix reserving for deleted targetJob raise nil pointer(https://github.com/volcano-sh/volcano/pull/1371, @zen-xu )
    • fix sla jobOderFn when sla not set(https://github.com/volcano-sh/volcano/pull/1365, @merryzhou )
    • fix: it is possible to Occur OutOfCpu， when exist some pods including init container(https://github.com/volcano-sh/volcano/pull/1364, @huone1 )
    • fix wrong Pipeline in action allocate(https://github.com/volcano-sh/volcano/pull/1360, @yzs981130 )
    • fix: prevent SelectBestNode func arise panic(https://github.com/volcano-sh/volcano/pull/1344, @yahaa )
    • fix(scheduler): move JobInfo helper functions to method(https://github.com/volcano-sh/volcano/pull/1343, @Thrimbda )
    Source code(tar.gz)
    Source code(zip)

  • v1.2.0(Feb 27, 2021)

    What's New

    1. Add TDM plugin

    TDM(Time Division Multiplexing) plugin aims to provide a mechanism for nodes, which can be used for K8S and other cluster(such as Yarn) in separate time.(https://github.com/volcano-sh/volcano/pull/1269, @yahaa )

    2. Add SLA plugin

    SLA(Service Level Agreement) plugin works for job resource reservation feature. Users can set SLA for jobs to ensure specified jobs to be scheduled in time. It provides an better design and implementation for job resource reservation. (https://github.com/volcano-sh/volcano/pull/1303, @jiangkaihua )

    Other Notable Changes

    • improve addResourceList func in job_controller_util.go(https://github.com/volcano-sh/volcano/pull/1332, @shinytang6 )
    • update overcommit plugin(https://github.com/volcano-sh/volcano/pull/1324, @jiangkaihua )
    • add e2e for sla plugin(https://github.com/volcano-sh/volcano/pull/1319, @jiangkaihua )
    • make sure non-preemptable and revocable workload not preempt other tasks in tdm plugin(https://github.com/volcano-sh/volcano/pull/1314, @wpeng102 )
    • support only specify preemptable=true for revocable workload(https://github.com/volcano-sh/volcano/pull/1313, @wpeng102 )
    • support revocable-zone annotaion for workload(https://github.com/volcano-sh/volcano/pull/1312, @wpeng102 )
    • add fail event for annotation admission(https://github.com/volcano-sh/volcano/pull/1308, @wpeng102 )
    • support min pod alive for tdm plugin(https://github.com/volcano-sh/volcano/pull/1300, @wpeng102 )
    • update enqueue action, import overcommit plugin to limit pending jobs from inqueue.(https://github.com/volcano-sh/volcano/pull/1298, @jiangkaihua )
    • build cache for revocable nodes(https://github.com/volcano-sh/volcano/pull/1293, @yahaa )
    • separate JobPipelined into two semantics for preempt action(https://github.com/volcano-sh/volcano/pull/1288, @wpeng102 )
    • support minAlive and evictMaxNum for job(https://github.com/volcano-sh/volcano/pull/1287, @wpeng102 )
    • non preemptable deployment preempt resource(https://github.com/volcano-sh/volcano/pull/1286, @wpeng102 )
    • update job-resource-reservation-design doc(https://github.com/volcano-sh/volcano/pull/1282, @Thor-wl )
    • add tdm design doc(https://github.com/volcano-sh/volcano/pull/1277, @wpeng102 )
    • refine deployment.yaml example(https://github.com/volcano-sh/volcano/pull/1274, @wpeng102 )
    • tdm plugin add victimsFn(https://github.com/volcano-sh/volcano/pull/1276, @wpeng102 )
    • add Makefile flag SUPPORT_PLUGINS(https://github.com/volcano-sh/volcano/pull/1266, @zen-xu )
    • update ssh secret when job updated(https://github.com/volcano-sh/volcano/pull/1263, @shinytang6 )
    • add job plugin example(https://github.com/volcano-sh/volcano/pull/1254, @shinytang6 )

    Bug Fixes

    • replace removed command of kind when getting kube config(https://github.com/volcano-sh/volcano/pull/1315, @rudeigerc )
    • fix log in job_controller_actions.go(https://github.com/volcano-sh/volcano/pull/1305, @gaocegege )
    • correct log info in cache.go(https://github.com/volcano-sh/volcano/pull/1302, @juchaosong )
    • optimize nodeorder plugin(https://github.com/volcano-sh/volcano/pull/1292, @huone1 )
    • enhance tdm max evict step(https://github.com/volcano-sh/volcano/pull/1290, @yahaa )
    • revert ssh subpath for ssh plugin(https://github.com/volcano-sh/volcano/pull/1280, @shinytang6 )
    • fix e2e helm install timeout(https://github.com/volcano-sh/volcano/pull/1262, @huone1 )
    • fix more pods are reclaimed than required(https://github.com/volcano-sh/volcano/pull/1260, @huone1 )
    • fix CI: add hacky retry mechanism(https://github.com/volcano-sh/volcano/pull/1248, @shinytang6 )
    Source code(tar.gz)
    Source code(zip)

  • v1.1.2(Feb 23, 2021)

    Changes since v1.1.1

    • bug fix: Use musl-gcc build image, because vc-scheduler default image is alpine, which only has musl-libc(https://github.com/volcano-sh/volcano/pull/1225, @zen-xu)
    Source code(tar.gz)
    Source code(zip)

  • v1.1.1(Dec 31, 2020)

    What's New

    1. support vc-scheduler loading custom plugins

    Separate plugin implementation with scheduler. Support implement custom plugins and load to vc-scheduler dynamically.(https://github.com/volcano-sh/volcano/pull/1218, @zen-xu)

    2. add MaxRequeueNum as a controller-manager param

    Support configure MaxRequeueNum in config file of vc-scheduler, default to 15 times.(https://github.com/volcano-sh/volcano/pull/1087, @shinytang6)

    3. add design documentation of CPU careful regulation

    Give the design of CPU careful regulation in socket level.(https://github.com/volcano-sh/volcano/pull/1051, @ProgramerGu)

    Other Notable Changes

    • add deployment example with volcano scheduling(https://github.com/volcano-sh/volcano/pull/1222, @Thor-wl)
    • add Queue & Namespace support for volcano monitoring(https://github.com/volcano-sh/volcano/pull/1200, @alcorj-mizar)
    • job event optimize (https://github.com/volcano-sh/volcano/pull/1192, @mikechengwei)
    • optimize local script(https://github.com/volcano-sh/volcano/pull/1191, @mikechengwei)
    • generate v1 crds. remove subresource status in pg crd(https://github.com/volcano-sh/volcano/pull/1179, @stpabhi)
    • mutate default of job spec(https://github.com/volcano-sh/volcano/pull/1170, @shinytang6)
    • CI switch to Github Action(https://github.com/volcano-sh/volcano/pull/1160, @daixiang0)
    • queue-resource design add(https://github.com/volcano-sh/volcano/pull/1158, @hudson741)
    • feature design for queue resource reservation(https://github.com/volcano-sh/volcano/pull/1130, @Thor-wl)
    • add podName and svcName length validate(https://github.com/volcano-sh/volcano/pull/1127, @mikechengwei)
    • update getting-started.md(https://github.com/volcano-sh/volcano/pull/1126, @daixiang0)
    • some improvements of scheduler(https://github.com/volcano-sh/volcano/pull/1111, @shinytang6)

    Bug Fixes

    • fix allocate action: high priority queue should not block others(https://github.com/volcano-sh/volcano/pull/1209, @yesterday)
    • fix script: fix daily release err(https://github.com/volcano-sh/volcano/pull/1208, @yesterday)
    • fix prepare-for-development dead link(https://github.com/volcano-sh/volcano/pull/1205, @naveensrinivasan)
    • fix gox path err im Makefile(https://github.com/volcano-sh/volcano/pull/1201, @shinytang6)
    • fix proportion can not reclaim issue(https://github.com/volcano-sh/volcano/pull/1194, @wpeng102)
    • fix dose not delete network policy when job finished(https://github.com/volcano-sh/volcano/pull/1186, @wpeng102)
    • fix queue cannot use idle resource issue(https://github.com/volcano-sh/volcano/pull/1176, @wpeng102)
    • fix scheduling duration update after complete(https://github.com/volcano-sh/volcano/pull/1167, @alcorj-mizar)
    • fix queue unknown state bug(https://github.com/volcano-sh/volcano/pull/1151, @wpeng102)
    • fix terminate job and release resources when drop job out of queue(https://github.com/volcano-sh/volcano/pull/1138, @merryzhou)
    • fix helm upgrade bugs on namespace hardcode(https://github.com/volcano-sh/volcano/pull/1132, @alcorj-mizar)
    Source code(tar.gz)
    Source code(zip)

  • v1.1.0(Oct 30, 2020)

    What's New

    1. Add monitor compontent

    Monitor compontent added support display some base metrics about Volcano.(https://github.com/volcano-sh/volcano/pull/1066, @alcorj-mizar)

    2. Support resource reservation for big job automatically

    Reserve resource for pending job which is at highest priority among pending jobs and waits for a long time. The big job is recognized by scheduler automatically.(https://github.com/volcano-sh/volcano/pull/1044, @Thor-wl)

    3. Support HDRF

    Hierarchical dominant resource fairness is configured with a weighted tree, such that each node in the tree has a positive weight value.(https://github.com/volcano-sh/volcano/pull/928, @ggaaooppeenngg)

    Other Notable Changes

    • add queue weight validation(https://github.com/volcano-sh/volcano/pull/1092, @Thor-wl)
    • add file watcher for scheduler config file(https://github.com/volcano-sh/volcano/pull/1016, @hzxuzhonghu)
    • add arm64 support(https://github.com/volcano-sh/volcano/pull/1000, @Thor-wl)

    Bug Fixes

    • not skip lower prioprity job when job with high priority allocated failed(https://github.com/volcano-sh/volcano/pull/1089, @merryzhou)
    • remove creationTimestamp comparation in jobOrderFn of gang plugin(https://github.com/volcano-sh/volcano/pull/1061, @zionwu)
    • fix queue capability overuse when specify minAvailable less than task replicas(https://github.com/volcano-sh/volcano/pull/1042, @zen-xu)
    • fix duplicate preemptee in victims(https://github.com/volcano-sh/volcano/pull/1023, @xiaoanyunfei)
    • fix scheduler panic when volcano job use pvc(https://github.com/volcano-sh/volcano/pull/1022, @wpeng102)
    • fix scheduler panic if minResource not set(https://github.com/volcano-sh/volcano/pull/1010, @hzxuzhonghu)
    • remove controlled resource in OnJobDelete(https://github.com/volcano-sh/volcano/pull/1005, @hzxuzhonghu)
    • fix ssh authorize key and remove no-root flag(https://github.com/volcano-sh/volcano/pull/996, @hzxuzhonghu)
    • plugins/binpack: fix typo of BinpackMemory(https://github.com/volcano-sh/volcano/pull/994, @aixeshunter)
    • scheduler: avoid pushing back empty jobs in allocate action(https://github.com/volcano-sh/volcano/pull/992, @lixiang233)
    • fix unallocate(https://github.com/volcano-sh/volcano/pull/984, @hzxuzhonghu)
    • fix bug of queue capability lose efficacy(https://github.com/volcano-sh/volcano/pull/974, @hzxuzhonghu)
    Source code(tar.gz)
    Source code(zip)
    volcano-v1.1.0-linux-gnu.tar.gz(66.95 MB)

  • v0.4.2(Aug 4, 2020)

  • v1.0.1(Jul 30, 2020)

  • v0.4.1(Jul 16, 2020)

  • v1.0.0(Jul 8, 2020)

    1.0 What's New

    1. GPU Sharing

    Volcano now supports gpu sharing between different pods (#852, @tizhou86, @hzxuzhonghu).

    2. Preempt and reclaim enhancement

    Volcano is now able to support preempt for batch job (#738, @carmark).

    3. Dynamic scale up and down

    Volcano job now supports dynamically scale up and down (#787, @hzxuzhonghu).

    4. Support integrate with flink operator

    Users are now able to run flink job with volcano. Follow the instructions here to make use of the feature. @hzxuzhonghu).

    5. Support DAG job with argo

    Users are now able to run DAG job with volcano. Follow the instructions here to make use of the feature. @alcorf-mizar).

    Other Notable Changes

    Bug Fixes

    Source code(tar.gz)
    Source code(zip)
    volcano-v1.0.0-linux-gnu.tar.gz(70.08 MB)

  • v0.4.0(Apr 9, 2020)

    v0.4.0 (2020-04-09)

    • #756 [controller] Set BlockOwnerDeletion to true when create PodGroup (@xiaogaozi)
    • #754 [controller] Set Queue field when pod has queue name annotation (@xiaogaozi)
    • #746 Fix volcano job phase setting (@hzxuzhonghu)
    • #745 Use go mod to manage dependencies (@tizhou86)
    • #733 Added resources predicate in allocate action (@k82cn)
    • #722 Added a flag disable-network-policy to disable Network Policy (@EDGsheryl)
    • #709 Use openssl to sign certificate instead of using k8s (@hzxuzhonghu)
    • #702 Added env var for scheduler name (@k82cn)
    • #693 Remove scheduling.v1alpha1 and scheduling.v1alpha2 API (@thandayuthapani)
    • #681 Refactor events/action (@k82cn)
    Source code(tar.gz)
    Source code(zip)

  • v0.4(Apr 7, 2020)

    v0.4 (2020-04-07)

    • #756 [controller] Set BlockOwnerDeletion to true when create PodGroup (@xiaogaozi)
    • #754 [controller] Set Queue field when pod has queue name annotation (@xiaogaozi)
    • #746 Fix volcano job phase setting (@hzxuzhonghu)
    • #745 Use go mod to manage dependencies (@tizhou86)
    • #733 Added resources predicate in allocate action (@k82cn)
    • #722 Added a flag disable-network-policy to disable Network Policy (@EDGsheryl)
    • #709 Use openssl to sign certificate instead of using k8s (@hzxuzhonghu)
    • #702 Added env var for scheduler name (@k82cn)
    • #693 Remove scheduling.v1alpha1 and scheduling.v1alpha2 API (@thandayuthapani)
    • #681 Refactor events/action (@k82cn)
    Source code(tar.gz)
    Source code(zip)

  • v0.3.0(Jul 28, 2020)

  • v0.3(Jan 21, 2020)

    • #670 Added Shareit as one of adopter (@k82cn)
    • #666 Add command vjobs, vqueues and unit tests. (@jiangkaihua)
    • #667 Revert "Gen install yaml without v1alpha queue and poggroup" (@hzxuzhonghu)
    • #664 Add capability in crd declarition (@hzxuzhonghu)
    • #663 add defaultQPS and defaultBurst for webhook-manager (@yuzhaojing)
    • #661 Change queue update in cli and e2e test to patch (@sivanzcw)
    • #660 Build new CLI by default. (@k82cn)
    • #656 Add vcommands: vcancel, vsuspend, vresume. (@jiangkaihua)
    • #658 Added scheduling v1beta1 API. (@k82cn)
    • #659 Add admission for queue (@sivanzcw)
    • #634 Do not create jobs until pg inqueue (@hzxuzhonghu)
    • #651 Reclaim Enhancement: Add Reclaimable parameter for queue (@sivanzcw)
    • #655 Auto generate code, change Copyright 2019 to Copyright 2020 (@sivanzcw)
    • #647 Considering best-effort pods when calculating ready task number (@sivanzcw)
    • #653 Gen install yaml without v1alpha queue and poggroup (@hzxuzhonghu)
    • #654 Remove pdb support (@hzxuzhonghu, @k82cn)
    • #652 Use relative path for doc. (@k82cn)
    • #633 fix the the Getting started link in contribute.md (@ruiyinchen)
    • #644 Refactor webhook org. (@k82cn)
    • #642 Added cherry_pick_pull.sh (@k82cn)
    • #643 Added Volcano Intro. (@k82cn)
    • #638 Remove duplicated check in jobEnqueueableFn of proportion (@zionwu)
    • #637 Update version to 0.3 (@k82cn)
    • #630 Added Roadmap (@k82cn)
    • #636 Modify error check of return error (@sivanzcw)
    • #631 Push job back to queue if task is assigned in reclaim action (@zionwu)
    • #632 remove redundant type conversion (@YesterdayxD)
    • #605 Rename binaries. (@k82cn)
    • #627 remove repeated code. (@YesterdayxD)
    • #626 Added Xiaohongshu as one of adopters (@k82cn)
    • #625 Update job_controller_util.go (@YesterdayxD)
    • #541 Pipeline task if task's request resource less than the releasing resource of node during performing allocate action (@sivanzcw)
    • #622 vcctl command line enhancement (@jiangkaihua)
    • #610 Added hosts into environment. (@k82cn)
    • #614 Update factory.go (@YesterdayxD)
    • #613 Added VC_TASK_INDEX and added env to initContainers. (@k82cn)
    • #609 Fixed build error of release-pkg. (@k82cn)
    • #608 Enahcement cli. (@k82cn)
    • #607 Fixed localup cluster script. (@k82cn)
    • #606 Update webhook path. (@k82cn)
    • #575 Admission Refactor. (@k82cn)
    • #603 change storage of ssh pem from configmap to secret for ssh plugin (@sivanzcw)
    • #601 Added localup script. (@k82cn)
    • #600 Remove kar, kube-batch. (@k82cn)
    • #599 Change lessequal function in Reclaimable function (@sivanzcw)
    • #597 when delete pod, a new shadowgroup will be created (@invalid-email-address)
    • #570 added priority based preemption to priority plugin (@mateuszlitwin)
    • #588 Cleanup e2e framework to speed up e2e (@hzxuzhonghu)
    • #591 disp job in default queue (@jiangkaihua)
    • #590 Support queue action by vcctl (@sivanzcw)
    • #589 Upgrade helm to v3.0.1 (@hzxuzhonghu)
    • #592 Add Vivo as adopter (@k82cn)
    • #587 Add arguments for action (@sivanzcw)
    • #585 use future idle resources when checking if task can fit node (@mateuszlitwin)
    • #512 Add queue controller about state (@sivanzcw)
    • #586 dep ensure (@sivanzcw)
    • #584 change node not found errors (@invalid-email-address)
    • #581 Change Statement unevict method to call UpdateTask (@yodarshafrir1)
    • #578 Add explict info about what todo to update generated yaml (@hzxuzhonghu)
    • #577 Enable CI verify (@hzxuzhonghu)
    • #576 Enable networkpolicy create/get permission (@hzxuzhonghu)
    • #572 fix validate victims check for preempt action (@zionwu)
    • #567 Update admission to use pflag. (@k82cn)
    • #564 Fixed build error. (@k82cn)
    • #566 Fix wrong condition for reclaim action (@zionwu)
    • #563 Update to klog. (@k82cn)
    • #542 modify the 'vcctl job run' function (@jiangkaihua)
    • #552 Support networkpolicy (@hzxuzhonghu)
    • #560 Move myself to controller owner (@hzxuzhonghu)
    • #547 Modify comments on OnPodCreate function of svc plugin (@sivanzcw)
    • #544 Simplify job pvc create process (@hzxuzhonghu)
    • #515 ssh plugin support specifying private/public keys path (@hzxuzhonghu)
    • #537 Add queueAction queueEvent queueRequest type (@sivanzcw)
    • #536 Add QTT as adopter (@k82cn)
    • #535 Add the --publish-not-ready-addresses param for the svc plugin (@zrss)
    • #527 Add svc hosts volumeMount for InitContainers (@zrss)
    • #525 Fixed import order. (@k82cn)
    • #523 pdb bug 修复 (@chenshaojin)
    • #520 Modify scheduling events for pod and podgroup (@sivanzcw)
    • #517 Add filter function for command watching of job controller (@sivanzcw)
    • #518 Added Gitter (@k82cn)
    • #507 fix filter NotReady node (@wangyuqing4)
    • #513 fix podgroup phase (@wangyuqing4)
    • #511 Umbrealla cleanups (@wangyuqing4)
    • #510 Rename imported package alias (@hzxuzhonghu)
    • #508 Add state parameter to queueSpec and queueStatus for queue (@sivanzcw)
    • #501 Add queue state management design proposal (@sivanzcw)
    • #506 Add events for pod with pipelined state (@sivanzcw)
    • #504 Dynamic loading comfigmap about action and plugins of scheduler, move loadSchedulerConf processing from run to runOnce (@sivanzcw)
    • #502 Fix deprecated dind in favor of kind in develop doc (@akillcool)
    • #499 correct podgroup creating bug for single pod without ownerreference (@sivanzcw)
    • #498 refresh volumes logic (@lminzhw, @dingtsh1)
    • #500 Request to be a reviewer (@yuanchen8911)
    • #497 add priorityClassName to podgroup during creating of podgroup from pod (@sivanzcw)
    • #491 Control the number of feasible nodes to find and score in scheduling (@yuanchen8911)
    • #494 format function name (@hzxuzhonghu)
    • #493 Added execution flow img. (@k82cn)
    • #490 Updated version to 0.2 (@k82cn)
    • #489 Added svg logo. (@k82cn)
    • #488 Admission: Fall back to v1alpha1 podgroup when v1alpha2 doesnot exist (@hzxuzhonghu)
    • #486 fix vvctl e2e (@hzxuzhonghu)
    • #485 Support KUBECON env. (@k82cn)
    • #478 check while ~/.kube/config is missing (@Rui-Tang)
    • #484 fix Resource Less/LessEqual (@wangyuqing4)
    • #482 fix proportion OverusedFn (@wangyuqing4)
    • #473 Comment out job volumes. (@k82cn)
    • #471 Rename file name of volcano intro in HC. (@k82cn)
    • #468 Added talks & integration into readme. (@k82cn)
    • #460 modify the return value of 'vcctl' (@jiangkaihua)
    • #463 Add HC demo. (@k82cn)
    • #451 Add Huawei-Cloud and GrandOmics (@k82cn)
    • #445 Queue refactor. (@k82cn)
    • #446 Register healthz interface for controller and scheduler (@sivanzcw)
    • #444 bump golang version to 1.13.x and kind to v0.5.0 in ci (@hzxuzhonghu)
    • #443 Add Baidu as adopter of Volcano. (@tizhou86)
    • #440 simplify README (@k82cn)
    • #442 Added adopters of Volcano. (@k82cn)
    • #439 set the json name of exitCode (@davidstack)
    • #437 Change image repository of mxnet demo from private to volcanosh (@sivanzcw)
    • #412 Add maxRetry in job controller to prevent endless loop (@hzxuzhonghu)
    • #411 Skip verify volcano job container's Privileged mode (@hzxuzhonghu)
    • #433 Fix CRD Definition (@hzxuzhonghu)
    • #434 Add demo about Click-Through-Rate distributed training with PaddlePad… (@sivanzcw)
    • #431 Moved KubeCon 2019 China demo to example. (@k82cn)
    Source code(tar.gz)
    Source code(zip)

  • v0.2(Sep 3, 2019)

    • #117 Implement queue Capability, donot allow podgroup enqueue when queue capability reached (@hzxuzhonghu)
    • #172 Show Queue's status in vkctl queue sub-command (@SrinivasChilveri)
    • #173 Add "vkctl job delete xxx" (@SrinivasChilveri)
    • #200 Disable preempt & relcaim action by default (@thandayuthapani)
    • #205 Check Queue exist in admission controller (@thandayuthapani)
    • #184 Use install job to generate secret for admission service (@TommyLike)
    • #149 Added Job garbage collector, cleanup Job after a configured ttl (@hzxuzhonghu)
    • #176 Retain pod with Succeeded/Failed phase (@lminzhw)
    • #170 Support Job Priority (@TommyLike)
    • #108 Resolve the golint issues (@nikita15p,@Rajadeepan)
    • #137 Pass conformance test (@shivramsrivastava)
    • #358 Fair-share scheduling of namespace cross queues (@lminzhw)
    • #306 Fix the scheduler panic whenever the GPU is lost on node (@william-wang)
    • #288 Migrate volcano-sh/scheduler into volcano-sh/volcano (@kevin-wangzefeng)
    • #168 Speed up e2e, do not just add an e2e test (@SrinivasChilveri,@thandayuthapani,@Rajadeepan)
    • #286 Update abbreviation of Volcano from vk-* to vc-*m, including binary and docker images (@asifdxtreme)
    • #93 Allow multiple sync job workers run in parallel (@SrinivasChilveri)
    • #329 Move admission webhook configuration registeration into admission server from yaml (@TommyLike)
    • #335 User experience improvement (@TommyLike)
    • #325 Contributor experience improvement (@hzxuzhonghu,@SrinivasChilveri)
    • #386 Valid policy action RestartTask is prevented (@hzxuzhonghu)
    • #266 Support multiple events in job lifecycle policy (@asifdxtreme)
    • #364 Migrate queue/podgroup to v1alpha2 (@hzxuzhonghu)
    • #401 Add PodGroupController to create shadow PodGroup (@wangyuqing4)
    • #384 The job task resync logic is not right (@hzxuzhonghu)
    • #370 Refactor Delay Pod Creation by admission controller (@wangyuqing4)
    • #380 Support binpack policy (@lminzhw)
    Source code(tar.gz)
    Source code(zip)
    volcano-0.2-linux-gnu.tar.gz(13.64 MB)

  • v0.1(May 14, 2019)

    Features

    • IndexedJob

    • Multiple Pod template

    • Error handling of Pod/Job

    • Queue/Job command line

    • Delay Pod Creation

    • Job plugins

      • env: set VK_TASK_INDEX to each container, is a index for giving the identity to container

      • svc: create headless Serivce and *.host to enable pods communicate

      • ssh: sign in ssh without password, e.g. use command mpirun or mpiexec

    Docker Images:

    docker pull volcanosh/vk-scheduler:v0.1
docker pull volcanosh/vk-controllers:v0.1
docker pull volcanosh/vk-admission:v0.1
    Source code(tar.gz)
    Source code(zip)
    vkctl-linux-amd64.tar.gz(11.67 MB)
Owner
Volcano
A Kubernetes Native Batch System
Volcano
GitHub https://volcano.sh
A native Go clean room implementation of the Porter Stemming algorithm.

Go Porter Stemmer A native Go clean room implementation of the Porter Stemming Algorithm. This algorithm is of interest to people doing Machine Learni

Charles Iliya Krempeaux 183 Jan 3, 2023
A recommender system service based on collaborative filtering written in Go

Language: English | 中文 gorse: Go Recommender System Engine Build Coverage Report GoDoc RTD Demo gorse is an offline recommender system backend based o

Zhenghao Zhang 6.4k Dec 29, 2022
Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.

English ∙ 日本語 ∙ 简体中文 ∙ 繁體中文 | العَرَبِيَّة‎ ∙ বাংলা ∙ Português do Brasil ∙ Deutsch ∙ ελληνικά ∙ עברית ∙ Italiano ∙ 한국어 ∙ فارسی ∙ Polski ∙ русский язы

Donne Martin 207.9k Jan 9, 2023
Go implementation of the yolo v3 object detection system

Go YOLO V3 This repository provides a plug and play implementation of the Yolo V3 object detection system in Go, leveraging gocv. Prerequisites Since

Wim Spaargaren 58 Dec 14, 2022
A Hackathon project created by Alpha Interface team for Agri-D Food Hack

Alpha Interface A Hackathon project created by Alpha Interface team for Agri-D Food Hack Installation Downloading Wasp and wasp-cli https://wiki.iota.

Jirawat Boonkumnerd 3 Oct 16, 2022
Kubernetes Native Edge Computing Framework (project under CNCF)

KubeEdge KubeEdge is built upon Kubernetes and extends native containerized application orchestration and device management to hosts at the Edge. It c

KubeEdge 5.5k Jan 1, 2023
OpenYurt - Extending your native Kubernetes to edge(project under CNCF)

openyurtio/openyurt English | 简体中文 What is NEW! Latest Release: September 26th, 2021. OpenYurt v0.5.0. Please check the CHANGELOG for details. First R

OpenYurt 1.4k Jan 7, 2023
Zero - If Google Drive says that 1 is under copyright, 0 must be under copyleft

zero Zero under copyleft license Google Drive's copyright detector says that fil

d!key 59 May 16, 2022
Fabric-Batch-Chaincode (FBC) is a library that enables batch transactions in chaincode without additional trusted systems.

Fabric-Batch-Chaincode Fabric-Batch-Chaincode (FBC) is a library that enables batch transactions in chaincode without additional trusted systems. Over

Datachain 1 Nov 9, 2022
GoBatch is a batch processing framework in Go like Spring Batch in Java

GoBatch English|中文 GoBatch is a batch processing framework in Go like Spring Batch in Java. If you are familiar with Spring Batch, you will find GoBat

null 21 Dec 25, 2022
dockin ops is a project used to handle the exec request for kubernetes under supervision

Dockin Ops - Dockin Operation service English | 中文 Dockin operation and maintenance management system is a safe operation and maintenance management s

WeBankFinTech 34 Aug 12, 2022
🐻 The Universal Service Mesh. CNCF Sandbox Project.

Kuma is a modern Envoy-based service mesh that can run on every cloud, in a single or multi-zone capacity, across both Kubernetes and VMs. Thanks to i

Kuma 2.3k Aug 10, 2021
🐻 The Universal Service Mesh. CNCF Sandbox Project.

Kuma is a modern Envoy-based service mesh that can run on every cloud, in a single or multi-zone capacity, across both Kubernetes and VMs. Thanks to i

Kuma 3k Jan 8, 2023
CNCF Jaeger, a Distributed Tracing Platform

Jaeger - a Distributed Tracing System Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing platform created by Uber Technologies and do

Jaeger - Distributed Tracing Platform 16.9k Jan 9, 2023
CNCF Jaeger, a Distributed Tracing Platform

Jaeger - a Distributed Tracing System Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing platform created by Uber Technologies and do

Jaeger - Distributed Tracing Platform 16.9k Jan 2, 2023
CNCF Jaeger, a Distributed Tracing Platform

Jaeger - a Distributed Tracing System Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing platform created by Uber Technologies and do

Jaeger - Distributed Tracing Platform 16.9k Jan 3, 2023
Project E-Commerce Alta Store Program Immersive Back End Batch 4

Project E-Commerce Project E-Commerce Alta Store Program Immersive Back End Batch 4 Explore the docs » Daftar Isi About The Project Built With ERD Con

null 1 Dec 30, 2021
A batch scheduler of kubernetes for high performance workload, e.g. AI/ML, BigData, HPC

kube-batch kube-batch is a batch scheduler for Kubernetes, providing mechanisms for applications which would like to run batch jobs leveraging Kuberne

Kubernetes SIGs 1k Jan 6, 2023
Set of Kubernetes solutions for reusing idle resources of nodes by running extra batch jobs

Caelus Caelus is a set of Kubernetes solutions for reusing idle resources of nodes by running extra batch jobs, these resources come from the underuti

Tencent 300 Nov 22, 2022
Cloud Native Electronic Trading System built on Kubernetes and Knative Eventing

Ingenium -- Still heavily in prototyping stage -- Ingenium is a cloud native electronic trading system built on top of Kubernetes and Knative Eventing

Mark Winter 6 Aug 29, 2022