Enterprise-Grade Continuous Delivery & DevOps Automation Open Source Platform

Overview

CDS: Continuous Delivery Service

Join the chat at https://gitter.im/ovh-cds/Lobby Go Report Card GoDoc

CDS is an Enterprise-Grade Continuous Delivery & DevOps Automation Platform written in Go(lang).

This project is under active development

Documentation

Intuitive UI

CDS provides an intuitive UI that allows you to build complex workflows, run them and dig into the logs when needed.

create and run workflow with CDS ui Create and run workflow with CDS ui.

The most powerful Command Line for a CI/CD Platform

cdsctl is the CDS Command Line - you can script everything with it, cdsctl also provide some cool commands such as cdsctl shell to browse your projects and workflows without the need to open a browser.

See all cdsctl commands

create workflow as code with CDS command line Create workflow as code with CDS command line.

Want a try?

Docker-Compose is your friend, see Ready To Run Tutorials

Blog posts and talks

FAQ

Why CDS? Discover the Origins

What is a CDS workflow?

Most of the CI/CD Tools play with jobs inside a pipeline. CDS introduce a new concept named CDS Workflows. A CDS Workflow allows you to chain pipelines with triggers. A pipeline is structured in sequential stages containing one or multiple concurrent jobs.

Can I use it in production?

Yes! CDS is used in production since 2015 @OVH and it launches more than 7M CDS workers per year. You can install the official release available on https://github.com/ovh/cds/releases

CDS provides everything needed to monitor and measure production activity (logs, metrics, monitoring)

How to backup?

All data are stored in the database - nothing on filesystem. Just backup your database regularly and you will be safe.

Need some help?

Core Team is available on Gitter

Comparison Matrix

All the features of the table are detailed below.

Feature CDS Bamboo Buildbot Gitlab CI Jenkins
Pipeline
Workflow *17
Visual configuration with Web UI *1
Configuration on Git Repository
Configuration as code on UI *2
Native Git branching *3
Job's Services
Secure Remote Caching *4
Enterprise Notification Bus & Built-in Hooks
Continuous Deployment & Environment Support *5
Enterprise-grade permissions, Support of ACLs delegation *6
Build Artifacts Cloud *7
Tests & Vulnerabilities Reports *8
Self-Service Project Creation - ability to create a tenant *9
Execution Environment Customization *10
Multi-Tenancy *11
Command Line Interface (cdsctl): 100% features supported & User Friendly *12
REST API & SDK
Self-Hosting
High Availability & Scalability & Upgrade without User Downtime *13
Built-in Metrics *14
Extensibility Plugins
OS/Arch Compatibility
Auto-Scale OnDemand multi-cloud *15 *16

CDS User features

Pipeline

Ability to run multiple jobs simultaneously while keeping an isolation between them. See doc about stages & jobs inside a pipeline. A pipeline is started with a context: 0 or 1 application, 0 or 1 environment.

Workflow

A Workflow makes it possible to chain the pipelines. This is a key Feature of CDS. You can create workflows using one or more pipelines, pipelines that can be linked together with joins or forks.

You can imagine having only one builder workflow and deploying your entire microservice stack. The same pipeline can be used several times in a workflow, you can associate an application, an environment. You will have only one deployment pipeline and only one build pipeline to maintain, even if you have hundreds of applications.

Workflow templates

A workflow template allow to share and reuse workflows across multiple teams. Any user can create a Workflow Template, maintain it as code or from UI, and bulk update a set of workflows with a single action.

As a company, you can offer a predefined catalog of workflows allowing you to standardize test and deployment practices across all your teams.

This also reduces the maintenance efforts since templates allow a scalable centralized management.

Visual configuration with Web UI

You can configure everything with the web UI. Even if you have complex use cases, it's usually easier to create your workflow graphically.

Configuration on Git Repository

Pipeline as code is a well-known concept of CI / CD tools. CDS, goes a step further and offers workflow as code. This is done by git-pushing using yaml configuration files of your workflow (+ pipeline, + applications, + environment). This is particularly useful as you can test your new workflow on a dev branch, before merging the changes on the master branch.

Configuration as code on UI

You can modify your workflow with the UI, you can also modify the configuration by editing the yml directly in the UI if you wish. This is an excellent way to learn how to use the workflow-as-code feature.

Native Git branching

Ability to launch builds based on a branch pattern. This allows, for example, to deploy dev/* branches to "staging" and deploy the master branch to "prod".

Note that CDS's default behavior is to launch the whole workflow on every git commit. This behavior can be altered using "run conditions".

Native GitHub / Bitbucket Server / GitLab / Gerrit integration

2-way integration with most popular git-based products.

  1. Ability to get notified and start a build when a change is pushed .
  2. Ability to notify the git-based tool of the success/failure of the build.

CDS natively supports GitHub, GitLab, Bitbucket Server and Gerrit. The link between your repo git and CDS is via a CDS application: 1 Git repository == a CDS application. Through this integration, CDS will push build status of your commits : Building, Success or Failed.

Multiple VCS Support in Pipeline/Job

CDS gives you the possibility to clone from different git repositories within a single workflow. A CDS workflow can involve several different applications - or none if you do not want to have a connection with a git repo.

Job's Services

Ability to start ephemeral services (a database, a web server, etc.) to support your job. This is particularly handy while testing your code.

In CDS these services are called Service Prerequisites. You just need to specify the corresponding docker image and run params.

Take a simple example: you have a pipeline that builds a docker image containing your application. Your application needs a redis and a postgreSQL to work. You can in a CDS job put three prerequisites service: a redis, a postgreSQL and your application. CDS will take care of making a private network between its services so that they can communicate with each other. Your CDS job can thus perform integration tests on your application started with a real database and a real cache.

Please read: https://ovh.github.io/cds/docs/concepts/requirement/requirement_service/

Secure Remote Caching

A remote cache is used by a team of developers and/or a continuous integration (CI) system to share build outputs. If your build is reproducible, the outputs from one machine can be safely reused on another machine, which can make builds significantly faster

Doc: https://ovh.github.io/cds/docs/components/worker/cache/

Enterprise Notification Bus

As an Enterprise-Grade platform, CDS can send a wide range of its internal events (e.g. build finished) in an event bus. This event flow can then feed other services (reporting, notifications, etc., ).

Built-in Hooks

Ability to launch a workflow manually or with git pushes or via a scheduler or via a webhook. In addition to the above, CDS can also be triggered using an event bus to (kafka or RabbitMQ).

Continuous Deployment & Environment Support

Ability to manage multiple environments (e.g. dev/prod/staging) in a secure way with segregated access rights. In practice, an environment is a just set of variables that you can use within your workflows.

With CDS, You can use a deployment pipeline on your preproduction environment and use that same deployment pipeline on your production environment. The ability to deploy to production can be limited to a pre-established group of users.

Enterprise-grade permissions / Support of ACLs delegation

Users are free to create groups and manage users in their groups. A group can have the rights to read, write, execute on their projects and their workflows. You can also restrict the execution of some pipelines to some groups if you wish.

Build Artifacts Cloud

If you use CDS as a CI / CD tool, you will probably have built artifacts. CDS jobs are isolated from each other, but you can pass artifacts from one job to another using the Artifact Upload and Artifact Download actions. At the end of a CDS job, all the files are deleted from the workers. To persist artifacts, CDS can use a Swift Storage or on a given filesystem (not recommended though).

Tests & Vulnerabilities Reports

CDS clearly displays the results of unit tests and vulnerabilities detected during builds.

Self-Service Project Creation

A CDS project is like a tenant. All users can create a CDS project, this project will bring together applications, environments, pipelines and of course workflows.

CDS projects are isolated from one another, but the same group may have access rights to multiple projects if you wish.

Execution Environment Customization

A worker model is a worker execution context. Let's say, you need to run a job that requires GoLang v1.11.5. In CDS, you just need to create a Go worker model, containing a go in version 1.11.5. A worker model can be a docker image, an openstack image, a VSphere image. Although CDS administrators can offer shared worker models, users can create their own template workers if they wish.

Self Service User’s Integrations

On a CDS project, you can add integrations like openstack, kubernetes, etc .... This will offer you features within your workflows. For example, with the Kubernetes integration, you can add your own cluster to your CDS project and thus be able to use the Deploy Application action to deploy your newly built application on your cluster, in helm format if you wish. You can of course develop your own integrations.

Multi-Tenancy

After reading the previous points, you've understood: self-service everywhere. All users can do their project / workflow / worker models / workflow templates / actions ... And run Jobs in a totally isolated environment. CDS projects are builders, on which you can add integrations. All this allows you to have only one instance of CDS for your entire company.

Command Line Interface (cdsctl): 100% features supported

All you can do with the UI is available via the Command-Line Interface (CLI), named "cdsctl". cdsctl is available on all the OS: darwin, freebsd, linux, openbsd ... cdsctl will allow you to create, launch, export, import your workflows, monitor your CDS, navigate through your projects, workflows. No need to go to the UI of CDS or your repository manager to check the status of your commit, git push && cdsctl workflow --track will display your workflow in your command line.

REST API & SDK

Do you have even more advanced automation needs, or the desire to develop an application that queries CDS? the REST API and the SDK will allow you to easily develop your software.

CDS Administration features

Self-Hosting

CDS is open-source since October 2016. You can install it freely in your company or at home. Some tutorials are available to help you start a CDS, docker-compose, Install with binaries.

High Availability / Scalability / Upgrade without User Downtime

High availability is a very important point for a CI / CD tool. CDS is stateless, nothing is stored on the filesystem. This makes it possible to launch several CDS APIs behind a load balancer. Thus, you can scale the API of CDS to your needs. It also allows upgrades of CDS in full day without impact for users. In production @OVH, CDS can be updated several times a day, without impacting users or stopping CDS workers. Asking your users to stop working while updating the Continuous Delivery tool would be ironic, isn't it? ;-)

Built-in Metrics

CDS natively exposes monitoring data. You will be able to feed your instance prometheus or warp10 using beamium.

Extensibility Plugins

A CDS job consists of steps. Each step is a built-in type action (script, checkoutApplication, Artifact upload / download ...). You can create your own actions, using existing actions - or develop your own action as a plugin. All languages are supported, as long as the language supports GRPC.

OS/Arch Compatibility

CDS is agnostic to languages and platforms. Users can launch Jobs on linux, windows, freebsd, osx, raspberry ... in Virtual Machine spawn on demand, in a docker container, on a dedicated host.

So, if your company uses multiple technologies, CDS will not be a blocker for building and deploying your internal software.

Auto-Scale OnDemand multi-cloud

One of the initial objectives of CDS at OVH: builder and deploy 150 applications as a container in less than 7 minutes. This has become reality since 2015. What is the secret key? Auto-Scale on Demand!

Thus, you can have hundreds of workers model and when necessary, CDS will start the workers using the hatcheries.

A hatchery is like an incubator, it gives birth to the CDS Workers and the right of life and death over them.

Several types of hatchery are available:

So yes, buzzwords or not, a multi-cloud Auto-scale OnDemand is a reality with CDS :-)

Some explanations on the comparison matrix

License

3-clause BSD

Comments
  • refactor/authentication (#4290)

    refactor/authentication (#4290)

    Description

    DOING

    • [x] Don't create authentified user while its local consumer has not been verified through it email address. @richardlt
    • [x] ui: edit "Patterns of configuration scripts" as admin. @richardlt
    • [x] ui: edit Worker model as admin. @richardlt
    • [ ] shared.infra should be removed from the worker consumer groups.
    • [x] api: update consumer if Ring change. @richardlt
    • [ ] doc: explain auth entities and scopes. @richardlt
    • [x] cdsctl: signup and reset (send mail that gives cdsctl instructions). @fsamin
    • [x] ui: component password strength, add it to reset page. @richardlt
    • [x] ui: password limit size (256). @richardlt
    • [x] cdsctl: merge refact context ctl #4576. @fsamin
    • [x] to merge #4573. @fsamin
    • [x] api: to encrypt data we should add some extra, use a value that allows to access the data (ex: key name). @fsamin
    • [x] instrumentation: router metrics. @fsamin
    • [x] ui: access /auth/signin should redirect if already signed in. @richardlt
    • [x] database downgrade
    • [x] user migration

    TODO

    • [x] Engine config file external key file management.
    • [x] update RUST client.

    TEST:

    • [x] it: create a test to init CDS to replace exiting scripting.
    • [x] it: test change group of a consumer should disabled or invalid part of it.
    • [x] it: clictl_access_token.yml
    • [x] it: clictl_action.yml
    • [x] it: clictl_admin_broadcast.yml
    • [x] it: clictl_admin_database.yml
    • [x] it: clictl_admin_integration_model.yml
    • [x] it: clictl_admin_metadata.yml
    • [x] it: clictl_admin_plugin_and_integrations.yml
    • [x] it: clictl_admin_services.yml
    • [x] it: clictl_common_options.yml
    • [x] it: clictl_group.yml
    • [x] it: clictl_project.yml
    • [x] it: clictl_template.yml
    • [x] it: clictl_template_apply.yml
    • [x] it: clictl_template_bulk.yml
    • [x] it: clictl_token.yml
    • [x] it: clictl_user.yml
    • [x] it: clictl_version.yml
    • [x] it: clictl_worker_model.yml
    • [x] it: sc_workflow_check_secret.yml
    • [x] it: sc_workflow_create_run_join.yml
    • [x] it: sc_workflow_create_simple.yml
    • [x] it: sc_workflow_edit_scheduler.yml
    • [x] it: sc_workflow_git_clone_tag.yml
    • [x] it: sc_workflow_integration_deploy_plugin_req.yml
    • [x] it: sc_workflow_key_install.yml
    • [x] it: sc_workflow_run_cache.yml
    • [x] it: sc_workflow_run_check_payload.yml
    • [x] it: sc_workflow_run_cmd_tmpl.yml
    • [x] it: sc_workflow_run_node_perm.yml
    • [x] it: sc_workflow_run_simple.yml
    • [x] it: sc_workflow_simple_serve.yml
    • [x] it: sc_workflow_stop_simple.yml
    • [x] it: sc_workflow_storage_awss3_artifact.yml
    • [x] it: sc_workflow_storage_awss3_worker_artifact.yml
    • [x] it: sc_workflow_storage_awss3_worker_cache.yml
    • [x] it: sc_workflow_storage_swift_artifact.yml
    • [x] it: sc_workflow_storage_swift_staticfiles.yml
    • [x] it: sc_workflow_storage_swift_worker_artifact.yml
    • [x] it: sc_workflow_storage_swift_worker_cache.yml
    • [x] it: sc_workflow_update_run.yml

    TO PLAN

    • [ ] cdsctl: signout (and clean context and secret)
    • [ ] cdsctl getting started: on first cdsctl run (without any context), run a login to configure all the things
    • [ ] api: update all requirements when renaming a group.
    • [ ] ui,api: add XSRF for signin local and ldap.
    • [ ] api: session lifetime should be based on last usage.
    • [ ] api: consumer "data" map should be scoped to authentication package.
    • [ ] Send mail on primary contact when a new consumer is added.
    • [ ] Active sessions managements, show all consulmers an session in an admin page.
    • [ ] instrumentation: Logger with session ID Field.
    • [ ] instrumentation: Expose TLS.
    • [ ] managed in memory cache of prepare statement in the gorpmapping layer
    • [ ] managed in memory cache of entities in the gorpmapping layer - to be used with sticky session on LB
    • [ ] remove deprecated tables

    DONE

    • [x] Test_checkWorkflowPermissionsByUser.
    • [x] ui: detach, delete, signin
    • [x] ui,api: regen consumers (should also invalidate old signin token).
    • [x] ui: show password not enough strong.
    • [x] api: username check syntax backend.
    • [x] api: consumer lifecycle. When a user is removed from a group, when a group is deleted, check if some consumers need to be revoked or updated.
    • [x] api: signin external should detect if there is a current consumer in session, then retrieve the user from it instead of username or email.
    • [x] api: implement session clean goroutine on API side.
    • [x] api: startup authentication drivers initialization.
    • [x] api: implement permActionBuiltinName.
    • [x] cdsctl: consumer & sessions management.
    • [x] cdsctl: login with external driver.
    • [x] cdsctl: if a signin token is given try to signin auto with builtin driver.
    • [x] LDAP authentication drivers
    • [x] api debug handlers getProfileIndexHandler
    • [x] updateGroupHandlerchecks number of remaining admins
    • [x] GetUserWorkflowEventssend email notifications- [x] postServiceRegisterHandler: for hatcheries, the user who created the used token must be admin of all groups in the token
    • [x] serviceAPIHeartbeat: fake user ?
    • [x] bookWorkerModelHandler: permModelID middleware
    • [x] Implement a unit test for checkGroupPermissions
    • [x] Implement a unit test for checkActionPermissions
    • [x] Implement a unit tests for checkTemplateSlugPermissions
    • [x] [ERROR] error executing query UPDATE services SET sig = $2 WHERE id = $1 with parameters services, id: pq: payload string too long
    • [x] Worker is revoked while building a job.
    • [x] admin: API symmecript rolling keys.
    • [x] list all TODOs in the code.
    • [x] ui,api: sse, send event based on permission need refacts. @fsamin
    • [x] getActionsForGroupHandler, getWorkerModelsForGroupHandler: move this in a permGroupName middleware, check usage of the handlers and change it. @richardlt
    • [x] Test_checkWorkerModelPermissionsByUser.
    • [x] hook: get 403 when at startup, check scopes. @fsamin
    • [x] Check the "triggered by" data when a workflow is run by a uService. @fsamin
    • [x] Implement a unit test for checkWorkerModelPermissions.
    • [x] secure the JWT cookie (http cookie attributes, path = "/"...). @richardlt
    • [x] services should stay up when register failed then retry (useful for cds init and upgrade). @fsamin
    • [x] test local hatchery
    • [x] test swarm hatchery
    • [x] test marathon hatchery
    • [x] test k8s hatchery
    • [x] test openstack hatchery
    • [x] ui: missing shared infra in new consumer modal. @richardlt
    • [x] api: handler /worker/model returns duplicate worker models
    • [x] Asynchronous handlers not working anymore.
    • [x] remove "communication with API" from worker model UI
    • [x] starting multiple hatcheries on the same engine failed: jobs_sse_count: cannot register view "jobs_sse_count"; a different view with the same name is already registered
    opened by richardlt 190
  • Refactor/users

    Refactor/users

    1. Description
    • [ ] When a user is removed from a group, removed from admins, when a group is deleted, check if some consumers need to be revoked or updated.
    • [ ] Use of a builtin signin token in CLI, should use the client that automatically signin. We can also signin manually.
    • [ ] Consumer "data" map should be scoped to authentication package.

    TODOs

    • [ ] cdsclt consumer & sessions management
    • [ ] getActionsForGroupHandler, getWorkerModelsForGroupHandler: move this in a permGroupID middleware
    • [x] api startup authentication drivers initialization
    • [x] LDAP authentication drivers
    • [x] api debug handlers getProfileIndexHandler
    • [x] updateGroupHandlerchecks number of remaining admins
    • [ ] GetUserWorkflowEventssend email notifications
    • [ ] Test_checkWorkerModelPermissionsByUser
    • [x] Test_checkWorkflowPermissionsByUser
    • [x] postServiceRegisterHandler: for hatcheries, the user who created the used token must be admin of all groups in the token
    • [x] serviceAPIHeartbeat: fake user ?
    • [x] bookWorkerModelHandler: permModelID middleware
    • [x] Implement a unit test for checkGroupPermissions
    • [ ] Implement a unit test for checkWorkerModelPermissions
    • [ ] Implement a unit test for checkActionPermissions
    • [ ] Implement permActionBuiltinName
    • [x] Implement a unit tests for checkTemplateSlugPermissions
    • [ ] Check the "triggered by" data when a workflow is run by a uService
    • [x] [ERROR] error executing query UPDATE services SET sig = $2 WHERE id = $1 with parameters services, id: pq: payload string too long
    • [x] Implement session clean goroutine on API side
    • [ ] Add XSRF signin local and ldap
    • [ ] username check syntax backend
    • [ ] Update all requirements that was using the group name
    refactor api 
    opened by fsamin 111
  • feat(cdsctl): cdsctl admin database for cdn service

    feat(cdsctl): cdsctl admin database for cdn service

    $ cdsctl admin database unlock api id-to-unlock $ cdsctl admin database unlock cdn id-to-unlock $ cdsctl admin database delete api id-migration-to-delete $ cdsctl admin database delete cdn id-migration-to-delete $ cdsctl admin database list api $ cdsctl admin database list cdn

    Signed-off-by: Yvonnick Esnault [email protected]

    opened by yesnault 19
  • feat(cdn): store job step logs

    feat(cdn): store job step logs

    • [x] Store logs from step
      • [x] Update Item when last logs
      • [x] Add api ref information in logs send by worker
    • [x] Store Sevice Logs
      • [x] Use the same algo
      • [x] Update Book Handler to have all api_ref data in hatchery
      • [x] Add api ref information in logs send by hatchery
      • [x] Append Logs
      • [x] Send Status information ( send done message when deleting the containers )
        • [x] Swarm send status
        • [x] Kube send status

    @ovh/cds

    opened by sguiheux 19
  • feat(api,ui): outgoing hooks

    feat(api,ui): outgoing hooks

    API:

    • [x] model definition
    • [x] CRUD Handlers
    • [x] hooks execution
    • [x] hooks triggers processing
    • [x] as code
    • [x] cds.status compute
    • [x] handle stop

    HOOKS:

    • [x] outgoing webhook execution
    • [x] outgoing workflow execution
    • [x] errors on execution
    • [x] interpolate variable on outgoing hook payload
    • [x] hide secrets in outgoing hook payload logs
    • [x] delete old executions

    UI:

    • [x] workflow edition (add and remove outgoing hooks)
    • [x] hooks edition (update outgoing hooks)
    • [x] run view
    • [x] hook run details (logs)
    • [x] hooks triggers
    • [x] link from child pipeline run to parent pipeline run
    opened by fsamin 19
  • feat: add version 2 of workflow export entities

    feat: add version 2 of workflow export entities

    1. Description
    • [x] Remove one pipeline mode
    • [x] Use a slice for notifications
    • [x] Display warning when load v1.0 workflow
    1. Related issues
    2. About tests
    3. Mentions

    @ovh/cds

    opened by sguiheux 17
  • Kafka integration with CDS failing

    Kafka integration with CDS failing

    Hi All, I am trying to integrate kafka with CDS, I have currently set up kafka in my host machine with single node in port 9092 as mentioned in official CDS documentation and also with CLI i am able to send and receive message but when I am trying to give the same configuration in CDS UI it is failing . I have attached the CDS UI screenshot and logs as well. Kindly let me know if i have missed out anything. Kafka_integration_error Kafka_integration_logs

    opened by shansclensky 14
  • Setup issues with release 0.40.0 in K8s and installation with Helm

    Setup issues with release 0.40.0 in K8s and installation with Helm

    HI Guys,

    Thanks for the new features in version 0.40.0

    Although, I can say that I'm having a very hard time to set CDS up no my k8s cluster. I'll describe my setup steps and point where thing gone wrong:

    1. I have a k8s cluster on aws-eks

    2. Helm, kubectl, tiller and aws cli installed and configured correctly on my Mac

    3. cloned the git repo - https://github.com/ovh/cds.git to my home directory

    4. inside ~/cds, git checkout 0.40.0. Now I'm in release 0.40. 0

    5. in contrib/helm/cds, installed CDS with helm helm dependency update helm install --name cds-weka . Issue 1: in helm/cds directory, tag values in values.yaml for cds image are 0.39.2, therefore, the installation resulted with older version - 0.39.2. 'app version' in Charts.yaml is also 0.39.2

    6. I deleted the helm installation- 'helm delete cds-weka --purge'

    7. modified values.yaml and Charts.yaml to the correct version -0.40.0

    8. helm dependency update helm install --name cds-weka . Now, version 0.40.0 is installed.

    9. added the bitbucketcloud section to config.toml, cluentid and secret and delete api and vcs pods. exactly as written in this guide: https://ovh.github.io/cds/docs/integrations/bitbucketcloud/ issue 2: open the cds ui--> enter a project advanced tab--> see no 'bitbucketcloud' in the 'Link to a repository manager' drop-down-list. I think there's missing documentation for how to enable the service.

    10. Go back to condig.toml- give a name to the 'name' field in 'CDS VCS Settings'- cds-weka-cds-vcs.

    11. kill api and vcs pods

    12. Finally, see bitbucketcloud in project's advanced tab.

    13. Connect to bitbucket--> 'grant access' in the bitbucket redirected page.

    14. Enter the code that is found url to 'validate' field in cds UI

    15. see the message - linked successfully to repository.

    16. now, trying to synchronise my repo list (drop down list) leads to: issue 3: the repositories drop down list is empty! authentication to my bitbucket cloud account probably not authorized...
      looking at the api pod logs, I see this: Reading configuration file config.toml 2019-07-02 07:32:36 [INFO] Directory /app/keys has been created 2019-07-02 07:32:36 [INFO] api> Service registered 2019-07-02 07:32:36 [INFO] Starting CDS API Server 0.40.0+cds.9951 2019-07-02 07:32:36 [INFO] Initializing mail driver... 2019-07-02 07:32:36 [INFO] Initializing feature flipping with izanami 2019-07-02 07:32:36 [INFO] Initializing local objectstore... 2019-07-02 07:32:36 [INFO] ObjectStore> Initialize Filesystem driver on directory: /app/artifacts 2019-07-02 07:32:36 [INFO] Initializing database connection... Starting service api 2019-07-02 07:32:36 [INFO] Bootstrapping database... 2019-07-02 07:32:36 [INFO] Initializing redis cache on weka-cds-devops-redis-master:6379... 2019-07-02 07:32:36 [INFO] Initializing HTTP router 2019-07-02 07:32:36 [INFO] Initializing Events broker 2019-07-02 07:32:36 [INFO] api> Stats initialized 2019-07-02 07:32:36 [INFO] Initializing Metrics 2019-07-02 07:32:36 [INFO] api> Metrics initialized 2019-07-02 07:32:36 [INFO] Initializing Authentication driver... 2019-07-02 07:32:36 [INFO] Auth> Initializing driver (local) 2019-07-02 07:32:36 [INFO] Auth> Connecting to session store 2019-07-02 07:32:36 [INFO] Initializing event broker... 2019-07-02 07:32:36 [INFO] Initializing internal routines... 2019-07-02 07:32:36 [INFO] Migration> CleanArtifactBuiltinActions> Already done (status: DONE) 2019-07-02 07:32:36 [INFO] Migration> WorkflowOldStruct> Already done (status: DONE) 2019-07-02 07:32:36 [INFO] Starting GRPC services on :8082 2019-07-02 07:32:36 [INFO] Migration> StageConditions> Already done (status: DONE) 2019-07-02 07:32:36 [INFO] Migration> WorkflowNotification> Already done (status: DONE) 2019-07-02 07:32:36 [INFO] Migration> ActionModelRequirements> Already done (status: DONE) 2019-07-02 07:32:36 [INFO] Starting CDS API HTTP Server on :8081 2019-07-02 07:35:06 [WARN] pprofLabel>recoverWrap>Handle>attachRepositoriesManagerHandler: repository not found (caused by: attachRepositoriesManager> Cannot get repo undefined: pprofLabel>recoverWrap>Handle>attachRepositoriesManagerHandler>RepoByFullname>doJSONRequest: resource not found (caused by: unable to get repo undefined from bitbucketcloud: pprofLabel>recoverWrap>Handle>attachRepositoriesManagerHandler>RepoByFullname>doJSONRequest>DoJSONRequest>doJSONRequest>ErrorWithFallback>NewErrorWithStack: wrong request (from: Unable to perform request on service weka-cds-devops-cds-vcs (vcs)) (caused by: Request Failed))) error_uuid=e9769dc1-9c9b-11e9-bd2e-76f3b50cd5a9 method=POST request_uri=/project/DEVOPTE/repositories_manager/bitbucketcloud/application/devops-test-app/attach?fullname=undefined stack_trace=attachRepositoriesManager> Cannot get repo undefined: pprofLabel>recoverWrap>Handle>attachRepositoriesManagerHandler>RepoByFullname>doJSONRequest: resource not found (caused by: unable to get repo undefined from bitbucketcloud: pprofLabel>recoverWrap>Handle>attachRepositoriesManagerHandler>RepoByFullname>doJSONRequest>DoJSONRequest>doJSONRequest>ErrorWithFallback>NewErrorWithStack: wrong request (from: Unable to perform request on service weka-cds-devops-cds-vcs (vcs)) (caused by: Request Failed)) github.com/ovh/cds/sdk.WrapError /go/src/github.com/ovh/cds/sdk/error.go:732 github.com/ovh/cds/engine/api.(*API).attachRepositoriesManagerHandler.func1 /go/src/github.com/ovh/cds/engine/api/repositories_manager.go:476 github.com/ovh/cds/engine/api.(*Router).Handle.func1 /go/src/github.com/ovh/cds/engine/api/router.go:245 net/http.HandlerFunc.ServeHTTP /usr/local/go/src/net/http/server.go:1995 github.com/ovh/cds/engine/api.(*Router).recoverWrap.func1 /go/src/github.com/ovh/cds/engine/api/router.go:152 net/http.HandlerFunc.ServeHTTP /usr/local/go/src/net/http/server.go:1995 github.com/ovh/cds/vendor/github.com/gorilla/handlers.CompressHandlerLevel.func1 /go/src/github.com/ovh/cds/vendor/github.com/gorilla/handlers/compress.go:143 net/http.HandlerFunc.ServeHTTP /usr/local/go/src/net/http/server.go:1995 github.com/ovh/cds/engine/api.(*Router).pprofLabel.func1 /go/src/github.com/ovh/cds/engine/api/router.go:89 net/http.HandlerFunc.ServeHTTP /usr/local/go/src/net/http/server.go:1995 github.com/ovh/cds/vendor/github.com/gorilla/mux.(*Router).ServeHTTP /go/src/github.com/ovh/cds/vendor/github.com/gorilla/mux/mux.go:114 net/http.serverHandler.ServeHTTP

    I tried many times since then, couldn't fix it. Please assist guys, thank you for your effort.

    opened by odedsCoolScripts 14
  • feat: allow lazy binary download

    feat: allow lazy binary download

    if there is no binary in docker image, there will be auto downloaded from github or artifactory if configured.

    Signed-off-by: Yvonnick Esnault [email protected]

    1. Description
    2. Related issues
    3. About tests
    4. Mentions

    @ovh/cds

    opened by yesnault 13
  • chore(deps): bump engine.io and socket.io in /ui

    chore(deps): bump engine.io and socket.io in /ui

    Bumps engine.io and socket.io. These dependencies needed to be updated together. Updates engine.io from 6.1.3 to 6.2.1

    Release notes

    Sourced from engine.io's releases.

    6.2.1

    :warning: This release contains an important security fix :warning:

    A malicious client could send a specially crafted HTTP request, triggering an uncaught exception and killing the Node.js process:

    Error: read ECONNRESET
        at TCP.onStreamRead (internal/stream_base_commons.js:209:20)
    Emitted 'error' event on Socket instance at:
        at emitErrorNT (internal/streams/destroy.js:106:8)
        at emitErrorCloseNT (internal/streams/destroy.js:74:3)
        at processTicksAndRejections (internal/process/task_queues.js:80:21) {
      errno: -104,
      code: 'ECONNRESET',
      syscall: 'read'
    }
    

    Please upgrade as soon as possible.

    Bug Fixes

    • catch errors when destroying invalid upgrades (#658) (425e833)

    6.2.0

    Features

    • add the "maxPayload" field in the handshake details (088dcb4)

    So that clients in HTTP long-polling can decide how many packets they have to send to stay under the maxHttpBufferSize value.

    This is a backward compatible change which should not mandate a new major revision of the protocol (we stay in v4), as we only add a field in the JSON-encoded handshake data:

    0{"sid":"lv_VI97HAXpY6yYWAAAC","upgrades":["websocket"],"pingInterval":25000,"pingTimeout":5000,"maxPayload":1000000}
    

    Links

    Changelog

    Sourced from engine.io's changelog.

    6.2.1 (2022-11-20)

    :warning: This release contains an important security fix :warning:

    A malicious client could send a specially crafted HTTP request, triggering an uncaught exception and killing the Node.js process:

    Error: read ECONNRESET
        at TCP.onStreamRead (internal/stream_base_commons.js:209:20)
    Emitted 'error' event on Socket instance at:
        at emitErrorNT (internal/streams/destroy.js:106:8)
        at emitErrorCloseNT (internal/streams/destroy.js:74:3)
        at processTicksAndRejections (internal/process/task_queues.js:80:21) {
      errno: -104,
      code: 'ECONNRESET',
      syscall: 'read'
    }
    

    Please upgrade as soon as possible.

    Bug Fixes

    • catch errors when destroying invalid upgrades (#658) (425e833)

    3.6.0 (2022-06-06)

    Bug Fixes

    Features

    • decrease the default value of maxHttpBufferSize (58e274c)

    This change reduces the default value from 100 mb to a more sane 1 mb.

    This helps protect the server against denial of service attacks by malicious clients sending huge amounts of data.

    See also: https://github.com/advisories/GHSA-j4f2-536g-r55m

    • increase the default value of pingTimeout (f55a79a)

    ... (truncated)

    Commits
    • 24b847b chore(release): 6.2.1
    • 425e833 fix: catch errors when destroying invalid upgrades (#658)
    • 99adb00 chore(deps): bump xmlhttprequest-ssl and engine.io-client in /examples/latenc...
    • d196f6a chore(deps): bump minimatch from 3.0.4 to 3.1.2 (#660)
    • 7c1270f chore(deps): bump nanoid from 3.1.25 to 3.3.1 (#659)
    • 535a01d ci: add Node.js 18 in the test matrix
    • 1b71a6f docs: remove "Vanilla JS" highlight from README (#656)
    • 917d1d2 refactor: replace deprecated String.prototype.substr() (#646)
    • 020801a chore: add changelog for version 3.6.0
    • ed1d6f9 test: make test script work on Windows (#643)
    • Additional commits viewable in compare view

    Updates socket.io from 4.4.1 to 4.5.3

    Release notes

    Sourced from socket.io's releases.

    4.5.3

    Bug Fixes

    • typings: accept an HTTP2 server in the constructor (d3d0a2d)
    • typings: apply types to "io.timeout(...).emit()" calls (e357daf)

    Links:

    4.5.2

    Bug Fixes

    • prevent the socket from joining a room after disconnection (18f3fda)
    • uws: prevent the server from crashing after upgrade (ba497ee)

    Links:

    4.5.1

    Bug Fixes

    • forward the local flag to the adapter when using fetchSockets() (30430f0)
    • typings: add HTTPS server to accepted types (#4351) (9b43c91)

    Links:

    4.5.0

    Bug Fixes

    • typings: ensure compatibility with TypeScript 3.x (#4259) (02c87a8)

    Features

    • add support for catch-all listeners for outgoing packets (531104d)

    This is similar to onAny(), but for outgoing packets.

    ... (truncated)

    Changelog

    Sourced from socket.io's changelog.

    4.5.3 (2022-10-15)

    Bug Fixes

    • typings: accept an HTTP2 server in the constructor (d3d0a2d)
    • typings: apply types to "io.timeout(...).emit()" calls (e357daf)

    4.5.2 (2022-09-02)

    Bug Fixes

    • prevent the socket from joining a room after disconnection (18f3fda)
    • uws: prevent the server from crashing after upgrade (ba497ee)

    2.5.0 (2022-06-26)

    Bug Fixes

    • fix race condition in dynamic namespaces (05e1278)
    • ignore packet received after disconnection (22d4bdf)
    • only set 'connected' to true after middleware execution (226cc16)
    • prevent the socket from joining a room after disconnection (f223178)

    4.5.1 (2022-05-17)

    Bug Fixes

    • forward the local flag to the adapter when using fetchSockets() (30430f0)
    • typings: add HTTPS server to accepted types (#4351) (9b43c91)

    4.5.0 (2022-04-23)

    Bug Fixes

    • typings: ensure compatibility with TypeScript 3.x (#4259) (02c87a8)

    ... (truncated)

    Commits
    • 945c84b chore(release): 4.5.3
    • d3d0a2d fix(typings): accept an HTTP2 server in the constructor
    • 19b225b docs(examples): update dependencies of the basic CRUD example
    • 8fae95d docs: add jsdoc for each public method
    • e6f6b90 docs: add deprecation notice for the allSockets() method
    • 596eb88 ci: upgrade to actions/[email protected] and actions/[email protected]
    • e357daf fix(typings): apply types to "io.timeout(...).emit()" calls
    • 10fa4a2 refactor: add list of possible disconnection reasons
    • 8be95b3 chore(release): 4.5.2
    • ba497ee fix(uws): prevent the server from crashing after upgrade
    • Additional commits viewable in compare view

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 1
  • chore(deps): bump loader-utils from 2.0.3 to 2.0.4 in /ui

    chore(deps): bump loader-utils from 2.0.3 to 2.0.4 in /ui

    Bumps loader-utils from 2.0.3 to 2.0.4.

    Release notes

    Sourced from loader-utils's releases.

    v2.0.4

    2.0.4 (2022-11-11)

    Bug Fixes

    Changelog

    Sourced from loader-utils's changelog.

    2.0.4 (2022-11-11)

    Bug Fixes

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 1
  • run-delete --no-interactive  still ask for confirmation

    run-delete --no-interactive still ask for confirmation

    When I launch:

    cdsctl workflow run-delete -n PROJECT my-workflow-name 315
    # or 
    cdsctl workflow run-delete --no-interactive PROJECT my-workflow-name 315
    

    it asks for confirmation

    ? Are you sure to delete? (Y/n)
    

    Version:

    ❯ cdsctl version
    cdsctl:
      version: 0.52.0
      architecture: arm64
      os: darwin
      git_hash: 1da97b5e45e1630f2da5b07c3e1789dd6d939e8d
      build_time: 09/26/22-14:58:49
      keychain: false
    api:
      version: 0.52.0-42-g20c5156.18205
      architecture: amd64
      os: linux
      git_hash: 20c51569bd1ffeae6f672a8941874cf4ee3a2e0c
      build_time: 11/08/22-10:10:46
      db_migrate: "0"
    
    opened by nicoovh 1
  • cdsctl pipeline export

    cdsctl pipeline export

    Hi. I have some pipelines with script action, and there are some bash strings with date format, git log format etc.

    For example: TZ=UTC-3 date +"%F %T" or git tag -l --format='%(refname:short) %(creatordate:short)'

    And when I'm doing ./cdsctl pipeline export $PROJECT $PIPELINE_NAME > file.yaml

    i get these strings in file like TZ=UTC-3 date +"%!F(MISSING) %!T(MISSING)"

    Is it a bug? Or how to deal with that?

    bug 
    opened by semenchukovps 1
  • CDS leaking DB connections

    CDS leaking DB connections

    Max DB connections is set to 20. After some running period I see DB connections is growing to more than 100. I'm using Postgres as DB.

    DB currently have a limit of 100, so after this max is reached CDS is hanging. The setup I'm using currently is:

    cds-engine start api ui cdn hooks vcs repositories hatchery:local so 1 process for all services. Local hatchery. max workers is set to 4.

    I do have a couple of workflows building my apps.

    attached a part of the logfile after a clean start.

    cds-engine.log

    bug 
    opened by ebo-bpi 1
Releases(0.52.0)
goveralls - Go integration for Coveralls.io continuous code coverage tracking system.

goveralls Go integration for Coveralls.io continuous code coverage tracking system. Installation goveralls requires a working Go installation (Go-1.2

mattn 746 Nov 24, 2022
An open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developersAn open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developers

Developer-oriented Continuous Delivery Product ⁣ English | 简体中文 Table of Contents Zadig Table of Contents What is Zadig Quick start How to use? How to

null 0 Oct 19, 2021
Enterprise-grade application development platform

Erda Overview Feature list Architecture Related repositories erda-proto erda-infra erda-ui Quick start To start using erda To start developing erda Do

Erda 2.4k Nov 21, 2022
Enterprise-grade container platform tailored for multicloud and multi-cluster management

KubeSphere Container Platform What is KubeSphere English | 中文 KubeSphere is a distributed operating system providing cloud native stack with Kubernete

KubeSphere 11.5k Nov 21, 2022
Drone is a Container-Native, Continuous Delivery Platform

Drone is a Continuous Delivery system built on container technology. Drone uses a simple YAML configuration file, a superset of docker-compose, to def

drone.io 26.1k Nov 26, 2022
Bubbly is an open-source platform that gives you confidence in your continuous release process.

Bubbly Bubbly - Release Readiness in a Bubble Bubbly emerged from a need that many lean software teams practicing Continuous Integration and Delivery

Valocode 34 Jul 27, 2022
KubeCube is an open source enterprise-level container platform

KubeCube English | 中文文档 KubeCube is an open source enterprise-level container platform that provides enterprises with visualized management of Kuberne

KubeCube IO 322 Nov 20, 2022
Continuous Delivery for Declarative Kubernetes, Serverless and Infrastructure Applications

Continuous Delivery for Declarative Kubernetes, Serverless and Infrastructure Applications Explore PipeCD docs » Overview PipeCD provides a unified co

PipeCD 634 Nov 28, 2022
Zadig is a cloud native, distributed, developer-oriented continuous delivery product.

Zadig Developer-oriented Continuous Delivery Product English | 简体中文 Table of Contents Zadig Table of Contents What is Zadig Quick start How to use? Ho

KodeRover 1.8k Nov 26, 2022
Zadig is a cloud native, distributed, developer-oriented continuous delivery product.

Zadig Developer-oriented Continuous Delivery Product ⁣ English | 简体中文 Table of Contents Zadig Table of Contents What is Zadig Quick start How to use?

KodeRover 30 May 12, 2021
Continuous Delivery with Spinnaker and Kubernetes

Continuous Delivery with Spinnaker and Kubernetes Test Result Tutorial Build App This tutorial takes you through the process of creating a reliable an

Laith 1 Dec 8, 2021
A Continuous Delivery system built on container technology

Drone is a Continuous Delivery system built on container technology. Drone uses a simple yaml configuration file, a superset of docker-compose, to def

null 0 Sep 29, 2021
Zeus - A Devops Automation Tool

With this tool we are trying generalize and minimize devops reperating task while trying to encourage shared responsibility model acorss devloper teams.

Intrigues 7 May 31, 2022
Open Source Continuous File Synchronization

Goals Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers. We strive to fulfill the goals belo

The Syncthing Project 47.8k Nov 18, 2022
Open Source Continuous File Synchronization

Goals Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers. We strive to fulfill the goals belo

The Syncthing Project 47.9k Nov 27, 2022
Devtron is an open source software delivery workflow for kubernetes written in go.

Devtron is an open source software delivery workflow for kubernetes written in go.

Devtron Labs 2.7k Nov 28, 2022
Apache Traffic Control is an Open Source implementation of a Content Delivery Network

Apache Traffic Control Apache Traffic Control is an Open Source implementation of a Content Delivery Network. Documentation Intro CDN Basics Traffic C

The Apache Software Foundation 833 Nov 18, 2022
The Bhojpur Wallet is a platform-as-a-service product used as a Wallet Engine based on the Bhojpur.NET Platform for application delivery.

Bhojpur Wallet - Data Processing Engine The Bhojpur Wallet is a platform-as-a-service used as a Service Engine based on the Bhojpur.NET Platform. It l

Bhojpur Consulting 1 Sep 26, 2022
An open-source and enterprise-level monitoring system.

Falcon+ Documentations Usage Open-Falcon API Prerequisite Git >= 1.7.5 Go >= 1.6 Getting Started Docker Please refer to ./docker/README.md. Build from

Open-Falcon 7k Nov 18, 2022
An open source alternative to terraform enterprise.

oTF An open source alternative to terraform enterprise. Functionality is currently limited: Remote execution mode (plans and applies run remotely) Sta

Louis Garman 99 Nov 20, 2022