Signed-off-by: Maria Kotlyarevskaya [email protected]

Hey @iximiuz! I took an initiative to implement some POC of CI for this repository. The main issue with it is that some of the examples expect you pass extra options during the start or create something beforehand.

The current implementation is very simple and works this way:

1. takes list of the directories in this repo, create a json valid list
2. run go mod tidy, go run main.go in each of this directories
3. it supports working with several go / kubernetes versions easily. (but we need to figure out how to set up it properly, because even now one of the examples requires go 1.18)

You can see it alive here. Let me know what do you think :)

I need code to create object from file just like kubectl since I don't want to depend on kubectl executable.

The following code works for creating Deployment, Service etc but not work for ClusterRole, ClusterRoleBind etc. The error is: client.Resource.Namespace.Create failed: the server could not find the requested resource.

Could you help to fix it?

package main

import (
	"context"
	"flag"
	"fmt"
	"io/fs"
	"os"
	"path/filepath"
	"sort"

	"github.com/thanos-io/thanos/pkg/errors"
	apiv1 "k8s.io/api/core/v1"
	apiErrors "k8s.io/apimachinery/pkg/api/errors"
	"k8s.io/apimachinery/pkg/api/meta"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
	"k8s.io/apimachinery/pkg/runtime/schema"
	"k8s.io/client-go/dynamic"
	"k8s.io/client-go/tools/clientcmd"
	"k8s.io/client-go/util/homedir"
	"sigs.k8s.io/yaml"
)

func kind2Resource(group, version, kind string) (gvr schema.GroupVersionResource, err error) {
	// https://www.cnblogs.com/zhangmingcheng/p/16128224.html
	// https://iximiuz.com/en/posts/kubernetes-api-structure-and-terminology/
	// https://github.com/kubernetes/kubernetes/issues/18622
	// kubectl api-resources
	gvk := schema.GroupVersionKind{Group: group, Version: version, Kind: kind}
	gvr, _ = meta.UnsafeGuessKindToResource(gvk)
	//fmt.Printf("%s %s %s -> %+v\n", group, version, kind, gvr)
	return
}

func createObject(client dynamic.Interface, fp string) (err error) {
	fmt.Printf("Reading %s...\n", fp)
	var b []byte
	if b, err = os.ReadFile(fp); err != nil {
		err = errors.Wrapf(err, "os.ReadFile failed")
		return
	}
	m := make(map[string]interface{})
	if err = yaml.Unmarshal(b, &m); err != nil {
		err = errors.Wrapf(err, "yaml.Unmarshal failed")
		return
	}

	obj := unstructured.Unstructured{Object: m}
	var apiVersion, kind, namespace string
	apiVersion = obj.GetAPIVersion()
	kind = obj.GetKind()
	namespace = obj.GetNamespace()
	if namespace == "" {
		namespace = apiv1.NamespaceDefault
	}
	gv, _ := schema.ParseGroupVersion(apiVersion)
	var gvr schema.GroupVersionResource
	if gvr, err = kind2Resource(gv.Group, gv.Version, kind); err != nil {
		return
	}
	var result *unstructured.Unstructured
	result, err = client.Resource(gvr).Namespace(namespace).Create(context.TODO(), &obj, metav1.CreateOptions{})
	if err != nil {
		err = errors.Wrapf(err, "client.Resource.Namespace.Create failed")
		return
	}
	fmt.Printf("Created resource %q.\n", result.GetName())
	return
}

func main() {
	// Initialize client
	var kubeconfig *string
	if home := homedir.HomeDir(); home != "" {
		kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
	} else {
		kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
	}
	pth := flag.String("f", ".", "path of manifest file or directory")
	flag.Parse()

	config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
	if err != nil {
		panic(err)
	}
	client, err := dynamic.NewForConfig(config)
	if err != nil {
		panic(err)
	}

	// Get manifest list
	var fi fs.FileInfo
	if fi, err = os.Stat(*pth); err != nil {
		panic(err)
	}
	var fps []string
	if fi.IsDir() {
		var des []fs.DirEntry
		if des, err = os.ReadDir(*pth); err != nil {
			panic(err)
		}
		for _, de := range des {
			if de.IsDir() {
				continue
			}
			fn := de.Name()
			ext := filepath.Ext(fn)
			if ext != ".yaml" && ext != "yml" {
				continue
			}
			fps = append(fps, filepath.Join(*pth, fn))
		}
		sort.Strings(fps)
	} else {
		fps = append(fps, *pth)
	}

	// Create resource for each manifest
	for _, fp := range fps {
		if err = createObject(client, fp); err != nil {
			if !apiErrors.IsAlreadyExists(err) {
				fmt.Println(err)
			}
		}
	}
}

# blackboxExporter-clusterRoleBinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: blackbox-exporter
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: 0.22.0
  name: blackbox-exporter
  namespace: monitoring
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: blackbox-exporter
subjects:
- kind: ServiceAccount
  name: blackbox-exporter
  namespace: monitoring

This is done a lot

home, err := os.UserHomeDir()
if err != nil {
	panic(err)
}

cfg, err := clientcmd.BuildConfigFromFlags("", path.Join(home, ".kube/config"))
if err != nil {
	panic(err.Error())
}

client := kubernetes.NewForConfigOrDie(cfg)
desired := corev1.ConfigMap{
	ObjectMeta: metav1.ObjectMeta{
		Name:      name,
		Namespace: namespace,
	},
	Data: map[string]string{"foo": "bar"},
}

Thoughts on a pkg/common or common/ or . I thin it will help better focus on the example that's trying to be displayed rather than the setup.

The repository should include an example of retry.RetryOnConflict.

Current implementation idea:

Folder Name: retry-on-conflict General Flow:

• Create pod - keep reference
• UpdateStatus pod - ignore reference to emulate the object changing unexpectedly from either another controller or external source
• Implement retry function. To display the use we could use a timer to demonstrate performing long running task to validate the status of an object. For instance sometimes when working with CRDs that create resources external to k8s, gathering status can take a significant amount of time. A very naive solution would be just using a flag and skipping fetch on the first try. Not quite sure, but a simple explanation would be better than a complex example. Maybe someone has some thoughts on how to emulate this situation easily.

err := retry.RetryOnConflict(retry.DefaultRetry, func() error {
    pod, err := c.Pods("ns").Get(name, metav1.GetOptions{})
    if err != nil {
        return err
    }
    
    // emulate long running tasks to get current status with timers
    
    _, err = c.Pods("ns").UpdateStatus(pod)
})

I'd like to submit a PR for this as I recently hit this position in a custom controller. @iximiuz I'd like to hear thoughts.

Thank you for the blogposts and for these snippets! I think it's very useful to see the timestamps in the examples that you have, I was wondering if you'd be ok if I submit a change replacing fmt.Printf with klog.Infof

So far:

I hope this is along the lines of what is expected.

Adds:

• simple display of retry on conflict. The current code will produce the following log

[email protected] retry-on-conflict (retry-on-conflict %)]$ go run main.go 
Successfully updated ConfigMap
Operation cannot be fulfilled on configmaps "foobar": the object has been modified; please apply your changes to the latest version and try again
Successfully updated ConfigMap
Operation cannot be fulfilled on configmaps "foobar": the object has been modified; please apply your changes to the latest version and try again
Successfully updated ConfigMap
Operation cannot be fulfilled on configmaps "foobar": the object has been modified; please apply your changes to the latest version and try again
Successfully updated ConfigMap
Operation cannot be fulfilled on configmaps "foobar": the object has been modified; please apply your changes to the latest version and try again
Successfully updated ConfigMap
Operation cannot be fulfilled on configmaps "foobar": the object has been modified; please apply your changes to the latest version and try again

I will add documentation on real world use cases/the nuances of the example in the readme. I will also be adding some docs about the function retryOnConflict itself and it's nuances.

cheers

Small typo

By the way awesome project. I find myself want to reference back to client-go example I no longer have access to or can't find. Expect to see a bit more of me.

Cheers!

At present, the selector remains empty even after the sel.Add(req) call.

fmt.Printf("%v", sel) prints empty string.

This is because the mutated sel is not reveived and assigned to sel at the caller.

note: the signature of sel.Add function is Add(r ...Requirement) Selector

Signed-off-by: Nikhil Thomas [email protected]