K3ai is a lightweight tool to get an AI Infrastructure Stack up in minutes not days.

Overview


Welcome to K3ai Project

K3ai is a lightweight tool to get an AI Infrastructure Stack up in minutes not days.

cli version  go version  go report  license


NOTE on the K3ai origins

Original K3ai Project has been developed at the end of October 2020 in 2 weeks by:

K3ai v1.0 has been entirely re-written by Alessandro Festa during the month of October 2021 to offer a better User Experience.

Thanks to the amazing and incredible people and projects that have been instrumental to create K3ai project repositories,website,etc...

โšก๏ธ Quick start

Let's discover K3ai in three simple steps.

๐ŸŒ˜ Getting Started

Get started by download k3ai from the release page here.

Or try K3ai companion script using this command:

curl -LO https://get.k3ai.in | sh -

๐ŸŒ— Load K3ai configuration

Let's start loading the configuration:

k3ai up

First time k3ai run will ask for a Github PAT (Personal Access Token) that we will use to avoid API calls limitations. Check Github Documentation to learn how to create one. Your personal GH PAT only need read repository permission.


๐ŸŒ– Configure the base infrastructure

Choose your favourite Kubernetes flavor and run it:

To know which K8s flavors are available

k3ai cluster list --all

it should print something like:

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ INFRASTRUCTURE                                                                                          โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ TYPE  โ”‚ DESCRIPTION                                         โ”‚ KIND  โ”‚ TAG    โ”‚ VERSION โ”‚ STATUS         โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ CIVO  โ”‚ The First Cloud Native Service Provider Power...    โ”‚ infra โ”‚ cloud  โ”‚ latest  โ”‚ Available      โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ EKS-A โ”‚ Amazon Eks Anywhere Is A New Deployment Option...   โ”‚ infra โ”‚ hybrid โ”‚ v0.5.0  โ”‚ Available      โ”‚
โ”‚       โ”‚ ate And Operate Kubernetes Clusters On Custome...   โ”‚       โ”‚        โ”‚         โ”‚                โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ K3S   โ”‚ K3s Is A Highly Available, Certified Kubernetes...  โ”‚ infra โ”‚ local  โ”‚ latest  โ”‚ Available      โ”‚
โ”‚       โ”‚ oads In Unattended, Resource-Constrained...         โ”‚       โ”‚        โ”‚         โ”‚                โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ KIND  โ”‚ Kind Is A Tool For Running Local Kubernetes...      โ”‚ infra โ”‚ local  โ”‚ v0.11.2 โ”‚ Available      โ”‚
โ”‚       โ”‚ as Primarily Designed For Testing Kubernetes...     โ”‚       โ”‚        โ”‚         โ”‚                โ”‚
โ”‚       โ”‚  Or Ci.                                             โ”‚       โ”‚        โ”‚         โ”‚                โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ TANZU โ”‚ Tanzu Community Edition Is A Fully-Featured...      โ”‚ infra โ”‚ hybrid โ”‚ latest  โ”‚ In Development โ”‚
โ”‚       โ”‚ ers And Users. It Is A Freely Available...          โ”‚       โ”‚        โ”‚         โ”‚                โ”‚
โ”‚       โ”‚  Of Vmware Tanzu.                                   โ”‚       โ”‚        โ”‚         โ”‚                โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

Now let start with something super fast and super simple:

k3ai cluster deploy --type k3s --n mycluster

๐ŸŒ Install a plugin to do your AI experimentations

Now that the server is up and running let's type:

k3ai plugin deploy -n mlflow -t mycluster

K3ai will print the url where you may access to the MLFLow tracking server at the end of the installation. That's all now just start having fun with K3ai!

๐ŸŒˆ Push a piece of code to the AI tools and focus on your goals

Let's push some code to the AI tool (i.e.: MLFlow)

k3ai run --source https://github.com/k3ai/quickstart --target mycluster --backend mlflow

wait the run to complete and login the backend AI tolls (i.e.: on the MLFlow UI http:// :30500 )

Current Implementation support

Operating Systems

Operating System K3ai v1.0.0
Linux Yes
Windows In Progress
MacOs In Progress
Arm In Progress

Clusters

K8s Clusters K3ai v1.0.0
Rancher K3s Yes
Vmware Tanzu Community Ed. Yes
Amazon EKS Anywhere Yes
KinD Yes

Plugins

Plugins K3ai v1.0.0
Kuebflow Components Yes
MLFlow Yes
Apache Airflow Yes
Argo Workflows Yes

โญ๏ธ Project assistance

If you want to say thank you or/and support active development of K3ai Project:

Together, we can make this project better every day! ๐Ÿ˜˜

โš ๏ธ License

K3ai is free and open-source software licensed under the BSD 3-Clause. Official logo was created by Alessandro Festa.

Issues
  • [Core] - Initial work for v3 version

    [Core] - Initial work for v3 version

    This PR is the initial work to re-write k3ai into a more flexible tool. This PR implements:

    • [x] #3
    • [x] #4
    • [x] #5
    • [ ] #6
    • [x] #7
    • [x] #8
    • [ ] #9

    This PR also include the Issues in the Plugin repo:

    • [x] https://github.com/k3ai/plugins/issues/1
    • [x] https://github.com/k3ai/plugins/issues/2
    • [x] https://github.com/k3ai/plugins/issues/3
    done 
    opened by alefesta 11
  • [BUG] k3ai up yields version `GLIBC_2.28' not found error

    [BUG] k3ai up yields version `GLIBC_2.28' not found error

    Describe the bug After following the installation instructions, the following error is reported:

    k3ai: /lib/x86_64-linux-gnu/libc.so.6: versionGLIBC_2.28' not found (required by k3ai)`

    To Reproduce Steps to reproduce the behavior:

    1. curl -LO https://get.k3ai.in | sh -
    2. k3ai up

    Expected behavior It should have spun up the cluster!

    OS: Ubuntu 18.04

    done bug 
    opened by htahir1 7
  • [Feature] - Running Kubeflow and MLFLow code through

    [Feature] - Running Kubeflow and MLFLow code through "One-Click" approach

    This PR address:

    • #10
    • #14 Also introduce -x as Extra and -e ad Entrypoint

    Examples:

    k3ai run -s https://github.com/alefesta/sample/mlflow -b mlflow -t <clustername>
    k3ai run -s https://github.com/alefesta/sample/kfp -b kfp -e condition.py -t <clustername>
    

    For MLFlow remains to manage the need of boto3 in the conda.yaml file that is a requirement to run on K8s. This need to be addressed before merge this PR. We may:

    • try to inject boto3 in the conda.yaml file at runtime
    • Force the user to fork the example first , change conda and run k3ai The first seems more compliant with k3ai goals of making life of user easier
    done 
    opened by alefesta 6
  • [BUG] - Kubeflow Pipelines Quickstart Repository Missing

    [BUG] - Kubeflow Pipelines Quickstart Repository Missing

    Describe the bug I was trying to follow the kubeflow pipelines tutorial as described in the k3ai website. It seems the final step of running the pipeline fails because the quickstart repository for kubeflow pipelines does not exist.

    To Reproduce Steps to reproduce the behavior:

    1. k3ai up
    2. k3ai cluster deploy -t k3s -n myk3scluster
    3. k3ai plugin deploy -n kf-pa -t myk3scluster
    4. k3ai run -s https://github.com/k3ai/quickstart/kfp -b kfp -e condition.py -t mycluster

    Expected behavior Pipeline to run successfully.

    Actual behavior Pipeline run fails.

    done 
    opened by harshitmahapatra 5
  • [Feature] - Add support for k3d

    [Feature] - Add support for k3d

    ๐Ÿš€ Is your feature request related to a problem? Please describe. Currently, k3ai doesn't have support for k3d.

    k3s is known to have issues with WSL2 deployment (systemd requirement, etc.), so it would be better to have k3d support.

    ๐Ÿ’ก Describe the solution you'd like We can add k3d support to k3ai in a subsequent release. (would require some work on pkg/io/execution).

    epic done 
    opened by burntcarrot 5
  • Fix lint issues

    Fix lint issues

    Fixed 100+ issues related to ineffectual assignments and added error checks. Added golangci-lint workflow to check linting issues while pushing code.

    The log level used while error checking is Fatal (log.Fatal(err)).

    opened by burntcarrot 4
  • [BUG] - MLFlow endpoint doesn't work in WSL2

    [BUG] - MLFlow endpoint doesn't work in WSL2

    Describe the bug

    While running the MLFlow plugin, the endpoint URI displayed by k3ai is not accessible.

    k3ai-mlflow

    The following endpoints are not accessible:

    • http://172.29.170.187:30500/ (displayed by k3ai)
    • http://172.29.170.187:5000/
    • http://10.96.150.194:30500/
    • http://10.244.0.7:30500/

    The IP address for the WSL2 machine is (through wsl hostname -I): 172.29.170.187

    WSL2 uses dynamic IP allocation.

    To Reproduce Steps to reproduce the behavior:

    k3ai run -s https://github.com/k3ai/quickstart -b mlflow
    

    Expected behavior The MLFlow endpoint exposed through k3ai should have worked.

    bug 
    opened by burntcarrot 4
  • [Feature] - Implement a system domain to automatically bind the plugins

    [Feature] - Implement a system domain to automatically bind the plugins

    K3ai should implement an automatice system domain (i.e: sslip.io or nip.io) so that any plugin installed could be exposed with the standard: <plugin-name>.<clusterIP>.nip.io This way we may use the same IP in cases like:

    • WSL
    • Laptops
    epic 
    opened by alefesta 4
  • [BUG] - runtime error with index out of range when running quickstart

    [BUG] - runtime error with index out of range when running quickstart

    Describe the bug A clear and concise description of what the bug is.

    To Reproduce Steps to reproduce the behavior:

    1. Follow quickstart steps:
    k3ai up
    k3ai cluster deploy -t k3s -n mycluster
    k3ai plugin deploy -n mlflow -t mycluster
    
    1. Try running quickstart: $ k3ai run -s https://github.com/k3ai/quickstart -b mlflow
    2. Receive error:
    ๐Ÿงช	Initializing code...
    panic: runtime error: index out of range [0] with length 0
    
    goroutine 1 [running]:
    github.com/k3ai/pkg/runner.Loader({0x7fff7fe7bb4e, 0x22}, {0x0, 0x0}, {0x7fff7fe7bb74, 0x6}, {0x0, 0x0}, {0x0, 0x0})
    	/home/joshec/git/k3ai/pkg/runner/run.go:78 +0x10f6
    github.com/k3ai/cmd.runCommand.func1(0xc000403680, {0xc0003d57c0, 0x0, 0x4})
    	/home/joshec/git/k3ai/cmd/run.go:71 +0x58b
    github.com/spf13/cobra.(*Command).execute(0xc000403680, {0xc0003d5780, 0x4, 0x4})
    	/home/joshec/go/pkg/mod/github.com/spf13/[email protected]/command.go:860 +0x5f8
    github.com/spf13/cobra.(*Command).ExecuteC(0x2254960)
    	/home/joshec/go/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3bc
    github.com/spf13/cobra.(*Command).Execute(...)
    	/home/joshec/go/pkg/mod/github.com/spf13/[email protected]/command.go:902
    github.com/k3ai/cmd.Execute(...)
    	/home/joshec/git/k3ai/cmd/root.go:34
    main.main()
    	/home/joshec/git/k3ai/main.go:10 +0x25
    

    Expected behavior A clear and concise description of what you expected to happen.

    A successful run with proper artifact storage and tracking URI settings

    Screenshots If applicable, add screenshots to help explain your problem.

    in progress bug 
    opened by jeinstei 3
  • [CI/CD] - Add Lint support

    [CI/CD] - Add Lint support

    On running golangci-lint on my local machine, I was able to find 40+ linting issues.

    10 of them were deadcode issues, so it can be ignored as they're a part of adding code for future releases.

    The rest are ineffectual assignments and skipped error checks. We can log the error message for the skipped error checks; it would help us more in debugging.

    I know this sounds like a minor issue, but with more code coming in the subsequent releases, addressing this earlier can help us save a lot of time maintaining good quality code.

    Suggested Fix: Add golangci-lint action as workflow to check linting issues. We can add a rule for excluding deadcode issues for now.

    done 
    opened by burntcarrot 3
  • 'invalid argument'

    'invalid argument'

    Hello - I am trying out the Mlflow deployment as in the tutorials and I get a stream of logs that say "invalid argument" and after a while I get "We tried to publish MLFLow at:http://172.17.0.2:30500" .. but when I go to this page there is no Mlflow server.

    Would appreciate the help. Thanks.

    Great work btw! this library is amazing!

    opened by jsnanavati 2
  • [BUG] - Kubeflow Pipelines not starting

    [BUG] - Kubeflow Pipelines not starting

    Describe the bug I am trying to run the kubeflow plugin on a single node 8vcpu / 16gb ram.

    To Reproduce curl -sfL https://get.k3ai.in | sh - k3ai up k3ai cluster deploy --type k3s -n mycluster k3ai plugin deploy -n kf-pa -t mycluster

    Issue Installation never ends, seems the pods are not being started correctly

    ubuntu:~$ k3s kubectl get pods -n kubeflow
    NAME                                              READY   STATUS                   RESTARTS        AGE
    workflow-controller-b7f95d6c6-q2wkf               1/1     Running                  0               4m22s
    ml-pipeline-scheduledworkflow-5c549bc5f5-drkmn    1/1     Running                  0               4m23s
    ml-pipeline-viewer-crd-7555c4d55f-fpd2m           1/1     Running                  0               4m23s
    metadata-envoy-deployment-7654b98955-rkt2g        1/1     Running                  0               4m24s
    ml-pipeline-ui-656466fdc9-qg9xv                   1/1     Running                  0               4m23s
    mysql-55778745b6-g4vbd                            1/1     Running                  0               4m22s
    minio-6d6d45469f-xgmz2                            1/1     Running                  0               4m24s
    cache-deployer-deployment-6f8ff5b986-tvwn4        1/1     Running                  0               4m24s
    metadata-grpc-deployment-5c8599b99c-b45jf         1/1     Running                  1 (3m17s ago)   4m24s
    ml-pipeline-8995b746f-dhznz                       1/1     Running                  1 (2m31s ago)   4m23s
    cache-server-74494cbf5-k956w                      0/1     Pending                  0               2m20s
    cache-server-74494cbf5-6v5lj                      0/1     ContainerStatusUnknown   0               4m24s
    ml-pipeline-persistenceagent-59689585f6-s8dhd     1/1     Running                  1 (2m5s ago)    4m23s
    ml-pipeline-visualizationserver-6b8fb8c44-mmrk8   0/1     ContainerStatusUnknown   0               4m22s
    ml-pipeline-visualizationserver-6b8fb8c44-svm25   0/1     Pending                  0               113s
    metadata-writer-fd965db48-9lw22                   0/1     Error                    0               4m24s
    metadata-writer-fd965db48-rqt7d                   0/1     Pending                  0               82s
    

    Pod metadata-writer-fd965db48-9lw22 error : message: 'The node was low on resource: ephemeral-storage. Container main was using 392Ki, which exceeds its request of 0. '

    Any ideas? Thanks!

    needs-triage bug 
    opened by tonxxd 0
  • [BUG] - postgress crashes when deploying mlflow on k3s / intel

    [BUG] - postgress crashes when deploying mlflow on k3s / intel

    Describe the bug in postgres pod: Bus error (core dumped)

    running on: (base) [email protected]:~$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04.4 LTS Release: 20.04 Codename: focal (base) [email protected]:~$

    To Reproduce

    k3ai cluster deploy --type k3s --name arrakis rk3ai plugin deploy -n mlflow -t arrakis k3s kubectl logs postgres-0

    Expected behavior successful mlflow startup

    Screenshots (base) [email protected]:~$ kubectl get all -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system pod/local-path-provisioner-6c79684f77-996dg 1/1 Running 0 7m28s kube-system pod/coredns-d76bd69b-8zqzz 1/1 Running 0 7m28s kube-system pod/metrics-server-7cd5fcb6b7-6zwwl 1/1 Running 0 7m28s default pod/minio-0 1/1 Running 0 6m40s default pod/mlflow-7c6768c4c-m6j6d 1/1 Running 0 6m23s default pod/postgres-0 0/1 CrashLoopBackOff 6 (20s ago) 6m31s

    NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default service/kubernetes ClusterIP 10.43.0.1 443/TCP 7m43s kube-system service/kube-dns ClusterIP 10.43.0.10 53/UDP,53/TCP,9153/TCP 7m40s kube-system service/metrics-server ClusterIP 10.43.39.68 443/TCP 7m39s default service/minio-service ClusterIP 10.43.144.140 9000/TCP 6m40s default service/postgres-service ClusterIP 10.43.236.158 5432/TCP 6m23s default service/mlflow-service NodePort 10.43.192.251 5000:30500/TCP 6m8s

    NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE kube-system deployment.apps/local-path-provisioner 1/1 1 1 7m40s kube-system deployment.apps/coredns 1/1 1 1 7m40s kube-system deployment.apps/metrics-server 1/1 1 1 7m39s default deployment.apps/mlflow 1/1 1 1 6m23s

    NAMESPACE NAME DESIRED CURRENT READY AGE kube-system replicaset.apps/local-path-provisioner-6c79684f77 1 1 1 7m29s kube-system replicaset.apps/coredns-d76bd69b 1 1 1 7m29s kube-system replicaset.apps/metrics-server-7cd5fcb6b7 1 1 1 7m29s default replicaset.apps/mlflow-7c6768c4c 1 1 1 6m23s

    NAMESPACE NAME READY AGE default statefulset.apps/minio 1/1 6m40s default statefulset.apps/postgres 0/1 6m31s

    (base) [email protected]:~$ kubectl logs postgres-0 The files belonging to this database system will be owned by user "postgres". This user must also own the server process.

    The database cluster will be initialized with locale "en_US.utf8". The default database encoding has accordingly been set to "UTF8". The default text search configuration will be set to "english".

    Data page checksums are disabled.

    fixing permissions on existing directory /var/lib/postgresql/mlflow/data ... ok creating subdirectories ... ok selecting default max_connections ... 20 selecting default shared_buffers ... 400kB selecting default timezone ... Etc/UTC selecting dynamic shared memory implementation ... posix creating configuration files ... ok

    bug todo :spiral_notepad: 
    opened by paxinos 1
  • [BUG] - Incompatible k3s version for kubeflow

    [BUG] - Incompatible k3s version for kubeflow

    Describe the bug There's bug on kubeflow part, they currently don't support k8s 1.22, so at the moment kubeflow pipelines seems to work, with k3s, but e.g. kf-dashboard is failing, which might be related to unsupported k8s API version.

    ...
     โณ     Working...
     ๐Ÿš€ unable to recognize "/home/user/.k3ai/git/common/istio-1-9/istio-crds/base": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
    ...
     ๐Ÿš€ unable to recognize "/home/user/.k3ai/git/common/istio-1-9/istio-install/base": no matches for kind "EnvoyFilter" in version "networking.istio.io/v1alpha3"
    ...
     ๐Ÿš€ unable to recognize "/home/user/.k3ai/git/common/istio-1-9/istio-install/base": no matches for kind "Gateway" in version "networking.istio.io/v1alpha3"
     ๐Ÿš€ unable to recognize "/home/user/.k3ai/git/common/istio-1-9/istio-install/base": no matches for kind "AuthorizationPolicy" in version "security.istio.io/v1beta1"
    ...
     ๐Ÿš€ unable to recognize "/home/user/.k3ai/git/common/istio-1-9/istio-install/base": no matches for kind "MutatingWebhookConfiguration" in version "admissionregistration.k8s.io/v1beta1"
     ๐Ÿš€ unable to recognize "/home/user/.k3ai/git/common/istio-1-9/istio-install/base": no matches for kind "ValidatingWebhookConfiguration" in version "admissionregistration.k8s.io/v1beta1"
    ...
    

    To Reproduce

    k3ai plugin deploy -n kf-dashboard -t myk3scluster

    Expected behavior

    Successful deployment of all kubeflow components on k3s.

    in progress docs 
    opened by Adiqq 1
  • [Feature] - Clean up CLI error messages

    [Feature] - Clean up CLI error messages

    ๐Ÿš€ Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

    See Issue https://github.com/k3ai/k3ai/issues/53 regarding k3ai run not returning a useful error when a required argument was missing

    ๐Ÿ’ก Describe the solution you'd like A clear and concise description of what you want to happen.

    a better CLI handler with more descriptive errors, or at least a fix for this bug on this command's handling

    ๐Ÿคฉ Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

    None yet

    help-wanted epic todo :spiral_notepad: docs 
    opened by jeinstei 0
  • Implement Plugin remove

    Implement Plugin remove

    Hey, great work with K3ai. Works For most of the operations pretty smooth.

    After experimenting a bit with Kubeflow I wanted to remove a plugin, but it seems that the command is not implemented:

    โžœ k3ai  plugin remove --name kf-pa
    Remove a given plugin based on NAME
    
    Usage:
      k3ai[options] plugin remove [-n NAME] [other flags]
    
    Flags:
      -n, --name string     NAME of plugin to be created/deleted
      -t, --target string   Target from where to remove plugin.
      -q, --quiet           Suppress output messages. Useful when k3ai is used within scripts.
      -c, --config string   Configure K3ai using a custom config file.[-c /path/tofile] [-c https://urlToFile]
    

    See here: https://github.com/k3ai/k3ai/blob/main/cmd/plugin.go#L138

    I am not sure, whether I am missing something but couldn't find anything related in the issues or Roadmap.


    • Your operating system name and version: Ubuntu 18.04
    • Detailed steps to reproduce the bug: Follow exact steps from documentation or README to deploy a plugin
    help-wanted epic todo :spiral_notepad: 
    opened by daniel-vera-g 2
  • [Feature] - Use Github Actions to create issues for exported reports

    [Feature] - Use Github Actions to create issues for exported reports

    ๐Ÿš€ Is your feature request related to a problem? Please describe. Related:

    • #37

    Using the metrics report exported through the executor, we can use Github Actions workflows to create automated issues containing the reports.

    ๐Ÿ’ก Describe the solution you'd like Create issues with the exported report as the content using Github Actions.

    epic todo :spiral_notepad: 
    opened by burntcarrot 1
Releases(1.0.1)
  • 1.0.1(Dec 7, 2021)

    Full Changelog: https://github.com/k3ai/k3ai/compare/1.0...1.0.1

    What's Changed

    K3ai Features:

    ARM support #13 @alefesta Kubeflow one-click pipeline #14 @alefesta Implementing GH actions (GHA) as a method to run K3ai from within the repo #19 Minimal documentation to run K3ai as GH @burntcarrot Implementing a config file to mimic an e2e workflow #18 @burntcarrot Add support for k3d #25 @burntcarrot

    K3ai Plugins:

    https://github.com/k3ai/plugins/issues/6 @alefesta https://github.com/k3ai/plugins/issues/7 @alefesta

    Bugs

    • [BUG] - certain plugins fail to install by @alefesta in https://github.com/k3ai/k3ai/pull/23
    • [BUG] - Fixes right tools download for Architecture by @alefesta in https://github.com/k3ai/k3ai/pull/43
    • [BUG] - minor fixes on download tools by @alefesta in https://github.com/k3ai/k3ai/pull/44
    • [BUG] - Fixes on Civo CLI for ARM by @alefesta in https://github.com/k3ai/k3ai/pull/45

    New Contributors

    • @burntcarrot made their first contribution in https://github.com/k3ai/k3ai/pull/27
    Source code(tar.gz)
    Source code(zip)
    k3ai(43.51 MB)
    k3ai.arm64(40.75 MB)
    k3ai.darwin.amd64(32.96 MB)
  • 1.0(Nov 1, 2021)

    Full Changelog: https://github.com/k3ai/k3ai/commits/1.0

    What's Changed

    • [Core] - Initial work for v1.0.0 version by @alefesta in https://github.com/k3ai/k3ai/pull/2

    New Contributors

    • @alefesta made their first contribution in https://github.com/k3ai/k3ai/pull/2

    Full Changelog:

    • Introducing K3ai DB to manage clusters and plugins dynamically
    • Introducing new CLI logic : K3ai [COMMAND] [ACTION] [OPTIONS]
    • Introducing the One Click experience to run training over deployed plugins.

    Current Operating Systems supported

    • Linux x64
    • macOS (Not Tested) Have fun with K3ai
    Source code(tar.gz)
    Source code(zip)
    k3ai(30.78 MB)
How fast could I write tic tac toe in Go, while not knowing Go, but with the aid of GitHub Copilot?

tictactoe-go-with-copilot How fast could I write tic tac toe in Go, while not knowing Go, but with the aid of GitHub Copilot? This took me about 30 mi

null 0 Dec 9, 2021
atwhy is a tool to describe your decisions inside the code where they are actually made and still get a readable documentation.

atwhy What is atwhy atwhy can be used to generate a documentation out of comments in the code. That way you can for example describe all available opt

Tiffinger & Thiel GmbH 5 Jan 29, 2022
Command line tool for improving typing skills (programmers friendly)

Command line tool for improving typing speed and accuracy. The main goal is to help programmers practise programming languages. Demo Installation Pyth

Jan 355 Aug 9, 2022
A tool for building identical machine images for multiple platforms from a single source configuration

Packer Packer is a tool for building identical machine images for multiple platforms from a single source configuration. Packer is lightweight, runs o

null 2 Oct 3, 2021
A quick and dirty hacked tool to query RIL100/DS100 abbreviations from the DB (Deutsche Bahn) in Germany.

go-ril100 A quick and dirty hacked tool to query RIL100/DS100 abbreviations from the DB (Deutsche Bahn) in Germany. usage text output $> go-ril100 DFL

Christian Richter 0 Jul 28, 2022
K3ai Executor is the runner pod to execute the "one-click" pipelines

Welcome to K3ai Project K3ai is a lightweight tool to get an AI Infrastructure Stack up in minutes not days. NOTE on the K3ai origins Original K3ai Pr

k3ai 0 Nov 11, 2021
himawari8-capturer is a tool that can get picture of the earth 20 minutes ago taken by the himawari 8

himawari8-capturer himawari8-capturer is a tool that can get picture of the earth taken by the himawari 8 Install Get the source code and build. How t

fissssssh 4 Jan 18, 2022
Package gostackparse parses goroutines stack traces as produced by panic() or debug.Stack() at ~300 MiB/s.

gostackparse Package gostackparse parses goroutines stack traces as produced by panic() or debug.Stack() at ~300 MiB/s. Parsing this data can be usefu

Datadog, Inc. 76 Jul 25, 2022
Elastic Stack Docker + Sample Go AppElastic Stack Docker + Sample Go App

?? Elastic Stack Docker + Sample Go App Test Elastic Stack which includes Elasticsearch, Kibana, Filebeat and Metricbeat. It comes with a very simple

Ruben Delgado 0 Jan 14, 2022
Distributed lock manager. Warning: very hard to use it properly. Not because it's broken, but because distributed systems are hard. If in doubt, do not use this.

What Dlock is a distributed lock manager [1]. It is designed after flock utility but for multiple machines. When client disconnects, all his locks are

Sergey Shepelev 25 Dec 24, 2019
Squizit is a simple tool, that aim to help you get the grade you want, not the one you have learnt for.

Squizit is a simple tool, that aim to help you get the grade you want, not the one you have learnt for. Screenshots First, input PIN Then enjoy! Hoste

cyan 11 Mar 11, 2022
null 38 Aug 3, 2022
Minutes is a CLI tool for synchronizing work logs between multiple time trackers, invoicing, and bookkeeping software to make entrepreneurs' daily work easier.

Minutes is a CLI tool for synchronizing work logs between multiple time trackers, invoicing, and bookkeeping software to make entrepreneurs' daily work easier.

Gรกbor Boros 21 Aug 8, 2022
Splasher is a tool to change desktop wallpaper random every 2 minutes from unsplash.

Splasher Splasher is a tool to change desktop wallpaper random every 2 minutes from unsplash. Download Latest Release: GitHub Release Installation & S

XXIV 0 Jan 7, 2022
Fast, lightweight and NOT reliable tool for downloading tons of images.

image-download-tool Fast, lightweight and NOT reliable tool for downloading tons of images. How to use Run .exe with --help flag Create json file with

Denis 3 May 12, 2022
Get subdomain list and check whether they are active or not by each response code. Using API by c99.nl

getsubdomain Get subdomain list and check whether they are active or not by each response code. Using API by c99.nl Installation โ–ถ go install github.c

Akbar Kustirama 9 Apr 30, 2022
7 days golang programs from scratch (web framework Gee, distributed cache GeeCache, object relational mapping ORM framework GeeORM, rpc framework GeeRPC etc) 7ๅคฉ็”จGoๅŠจๆ‰‹ๅ†™/ไปŽ้›ถๅฎž็Žฐ็ณปๅˆ—

7 days golang programs from scratch README ไธญๆ–‡็‰ˆๆœฌ 7ๅคฉ็”จGoไปŽ้›ถๅฎž็Žฐ็ณปๅˆ— 7ๅคฉ่ƒฝๅ†™ไป€ไนˆๅ‘ข๏ผŸ็ฑปไผผ gin ็š„ web ๆก†ๆžถ๏ผŸ็ฑปไผผ groupcache ็š„ๅˆ†ๅธƒๅผ็ผ“ๅญ˜๏ผŸๆˆ–่€…ไธ€ไธช็ฎ€ๅ•็š„ Python ่งฃ้‡Šๅ™จ๏ผŸๅธŒๆœ›่ฟ™ไธชไป“ๅบ“่ƒฝ็ป™ไฝ ็ญ”ๆกˆ

Dai Jie 11.1k Aug 8, 2022
Source code and slides for Kubernetes Community Days - Bangalore.

kcdctl This is the source code for the demo done as part of the talk "Imperative, Declarative and Kubernetes" at the Kubernetes Community Days, Bengal

Madhav Jivrajani 15 Sep 19, 2021
100 days of Go learning

This repository is a journal of my path to learning GO. By the end of the 100 days, you should be able to follow along by day and learn Go as well.

Cobra 24 Feb 13, 2022
Copy of stdlib's time.Duration, but ParseDuration accepts other bigger units such as days, weeks, months and years

duration Copy of stdlib's time.Duration, but ParseDuration accepts other units as well: d: days (7 * 24 * time.Hour) w: weeks (7 * Day) mo: months (30

Carlos Alexandro Becker 11 Jun 21, 2022
Birthdays is a web service that stores the birthday date of users and calculates the remaining days until the user's birthday.

Birthdays is a web service that stores the birthday date of users and calculates the remaining days until the user's birthday. Features Metrics servic

Hugo Meneses 1 May 2, 2022
This repository will have code implemented for the 100 days of golang.

golang_100 This repository will have code implemented for the 100 days of golang. The resources I will use to do this 100 days golang programming are:

Vasileios Tsakalos 0 Jan 10, 2022
7 days golang programs from scratch

7 days golang programs from scratch README ไธญๆ–‡็‰ˆๆœฌ 7ๅคฉ็”จGoไปŽ้›ถๅฎž็Žฐ็ณปๅˆ— 7ๅคฉ่ƒฝๅ†™ไป€ไนˆๅ‘ข๏ผŸ็ฑปไผผ gin ็š„ web

quickler 0 Dec 18, 2021
Day-1 is apart of my 6 days of Christmas challenge where i write in two new languages everyday, and make something weird out of it.

Day-1 is apart of my 6 days of Christmas challenge where i write in two new languages everyday, and make something weird out of it. today was a HTTP server written with PostGreSQL using Golang, R, and shell script read more

RE43P3R 0 Dec 21, 2021
Goodmorning - The number of days left in the year and the weather for today

goodmorning ไปŠๅนดใฎๆฎ‹ใ‚Šใฎๆ—ฅๆ•ฐใจใŠๅคฉๆฐ—ใŒใ‚ใ‹ใ‚‹ใ‚ˆ ๅฎŸ่กŒใ‚ตใƒณใƒ—ใƒซ โ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒโ–ฝโ–ฒ

awukomaki 1 Feb 16, 2022
A Github action that notifies PR's that are open longer than X days

PR notifier Use GitHub Actions to notify Slack that a pull request is opened since DAYS_BEFORE days. Usage Add the following YAML to your new GitHub A

Movingimage EVP 6 Apr 8, 2022
Higher Order Functions using Golang Generics (Hack Days 2022)

hoff: Higher Order Functions (and Friends) Golang 1.18+ implementations of common methods/data structures using Go Generics Requirements Go 1.18 or ne

Shopify 16 Jul 29, 2022
Walrus - Fast, Secure and Reliable System Backup, Set up in Minutes.

Walrus is a fast, secure and reliable backup system suitable for modern infrastructure. With walrus, you can backup services like MySQL, PostgreSQL, Redis, etcd or a complete directory with a short interval and low overhead. It supports AWS S3, digitalocean spaces and any S3-compatible object storage service.

Ahmed 443 Aug 11, 2022
GraphJin - Build APIs in 5 minutes with GraphQL. An instant GraphQL to SQL compiler.

GraphJin - Build APIs in 5 minutes GraphJin gives you a high performance GraphQL API without you having to write any code. GraphQL is automagically co

Vikram Rangnekar 2.3k Aug 8, 2022