Standard machine learning models

Related tags

Machine Learning cog
Overview

Cog: Standard machine learning models

Define your models in a standard format, store them in a central place, run them anywhere.

  • Standard interface for a model. Define all your models with Cog, in a standard format. It's not just the graph – it also includes code, pre-/post-processing, data types, Python dependencies, system dependencies – everything.
  • Store models in a central place. No more hunting for the right model file on S3. Cog models are in one place with a content-addressable ID.
  • Run models anywhere: Cog models run anywhere Docker runs: your laptop, Kubernetes, cloud platforms, batch processing pipelines, etc. And, you can use adapters to convert the models to on-device formats.

Cog does a few things to make your life easier:

  • Automatic Docker image. Define your environment with a simple format, and Cog will generate CPU and GPU Docker images using best practices and efficient base images.
  • Automatic HTTP service. Cog will generate an HTTP service from the definition of your model, so you don't need to write a Flask server in the right way.
  • No more CUDA hell. Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combos are compatible and will pick the right versions for you.

How does it work?

  1. Define how inferences are run on your model:
import cog
import torch

class ColorizationModel(cog.Model):
    def setup(self):
        self.model = torch.load("./weights.pth")

    @cog.input("input", type=Path, help="Grayscale input image")
    def run(self, input):
        # ... pre-processing ...
        output = self.model(processed_input)
        # ... post-processing ...
        return processed_output
  1. Define the environment it runs in with cog.yaml:
model: "model.py:ColorizationModel"
environment:
  python_version: "3.8"
  python_requirements: "requirements.txt"
  system_packages:
   - libgl1-mesa-glx
   - libglib2.0-0
  1. Push it to a repository and build it:
$ cog build
--> Uploading '.' to repository http://10.1.2.3/colorization... done
--> Building CPU Docker image... done
--> Building GPU Docker image... done
--> Built model b6a2f8a2d2ff

This has:

  • Created a ZIP file containing your code + weights + environment definition, and assigned it a content-addressable SHA256 ID.
  • Pushed this ZIP file up to a central repository so it never gets lost and can be run by anyone.
  • Built two Docker images (one for CPU and one for GPU) that contains the model in a reproducible environment, with the correct versions of Python, your dependencies, CUDA, etc.

Now, anyone who has access to this repository can run inferences on this model:

$ cog infer b6a2f8a2d2ff -i @input.png -o @output.png
--> Pulling GPU Docker image for b6a2f8a2d2ff... done
--> Running inference... done
--> Written output to output.png

It is also just a Docker image, so you can run it as an HTTP service wherever Docker runs:

$ cog show b6a2f8a2d2ff 
...
Docker image (GPU):  registry.hooli.net/colorization:b6a2f8a2d2ff-gpu
Docker image (CPU):  registry.hooli.net/colorization:b6a2f8a2d2ff-cpu

$ docker run -d -p 8000:8000 --gpus all registry.hooli.net/colorization:b6a2f8a2d2ff-gpu

$ curl http://localhost:8000/infer -F [email protected]

Why are we building this?

It's really hard for researchers to ship machine learning models to production. Dockerfiles, pre-/post-processing, API servers, CUDA versions. More often than not the researcher has to sit down with an engineer to get the damn thing deployed.

By defining a standard model, all that complexity is wrapped up behind a standard interface. Other systems in your machine learning stack just need to support Cog models and they'll be able to run anything a researcher dreams up.

At Spotify, we built a system like this for deploying audio deep learning models. We realized this was a repeating pattern: Uber, Coinbase, and others have built similar systems. So, we're making an open source version.

The hard part is defining a model interface that works for everyone. We're releasing this early so we can get feedback on the design and find collaborators. Hit us up if you're interested in using it or want to collaborate with us. We're on Discord or email us at [email protected].

Install

No binaries yet! You'll need Go 1.16, then run:

make install

This installs the cog binary to $GOPATH/bin/cog.

Next steps

Comments
  • No CUDA runtime is found / Found no NVIDIA driver on your system.

    No CUDA runtime is found / Found no NVIDIA driver on your system.

    When running a GPU cog:

    $ sudo cog predict r8.im/allenhung1025/[email protected]:f5cd715e99046e0513fe2b4034e8f7d8c102525b02f49efb52b05f46fcb9ea83
    Starting Docker image r8.im/allenhung1025/[email protected]:f5cd715e99046e0513fe2b4034e8f7d8c102525b02f49efb52b05f46fcb9ea83 and running setup()...
    No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
    Traceback (most recent call last):
    ...
    AssertionError: 
    Found no NVIDIA driver on your system. Please check that you
    have an NVIDIA GPU and installed a driver from
    http://www.nvidia.com/Download/index.aspx
    ⅹ Failed to get container status: exit status 1
    

    But CUDA is fine:

    $ cd /usr/local/cuda/samples/1_Utilities/deviceQuery
    $ sudo make
    $ ./deviceQuery
    Detected 1 CUDA Capable device(s)
    
    Device 0: "NVIDIA GeForce RTX 2080 Ti"
      CUDA Driver Version / Runtime Version          11.4 / 11.2
      CUDA Capability Major/Minor version number:    7.5
      Total amount of global memory:                 11016 MBytes (11551440896 bytes)
      (68) Multiprocessors, ( 64) CUDA Cores/MP:     4352 CUDA Cores
    ...
    Result = PASS
    
    opened by bwhitman 12
  • cog: add support for

    cog: add support for "environment" to set environment variables in the generated Dockerfile

    ℹ️ Docs preview: https://github.com/hangtwenty/cog/blob/cog-environment-variables/docs/yaml.md#environment


    Change 1/2: support for environment (ENV)

    Closes issue #291, "Environment variables in cog.yaml." In Dockerfile parlance, it's support for ENV. This supersedes my the previous PR, #361.

    This enables adding environment to build in cog.yaml:

    build:
      environment:
        - EXAMPLE=/src/example
        - DEBUG=1
    

    And those become ENV directives in the Dockerfile.

    Change 2/2: A single default environment variable to configure caching for PyTorch and some other libraries (XDG_CACHE_HOME)

    One of the libraries used most often with cog is PyTorch. You don't have to do configure anything for it, and now the caching should "just work" in most cases. Likewise for other popular libraries.

    Variables you DON'T need to set

    You don't need to set any of the following because the sane default for $XDG_CACHE_HOME will take care of it. (Because it's part of a standard, several libraries use it for their defaults.)

    • pytorch: You don't need to set PYTORCH_TRANSFORMERS_CACHEdocs
    • Hugging Face: You don't need to set TRANSFORMERS_CACHEdocs
      • You shouldn't have to set PYTORCH_PRETRAINED_BERT_CACHEdocs
    • pip: no variables needed! Before this PR, cog already configured the pip/apt-get cache directories

    You don't need to set TORCH_HOME

    If TORCH_HOME is set, that'll take precedence over XDG_CACHE_HOME — see pytorch docs, huggingface docs. So, if you set TORCH_HOME to something else, just be sure to set it within /src/, like /src/.cache-torch. That way, it will get cached across runs.

    opened by floer32 10
  • Refactor `Makefile`

    Refactor `Makefile`

    This PR implements some changes I needed to get a Homebrew formula working (see https://github.com/replicate/cog/issues/822), specifically:

    • Defining constants for executables like PYTHON. When Homebrew installs Python 3, my understanding is that it links to python3, keeping the default python (2.7) executable accessible in the default PATH. Because Python 3 is a prerequisite for building from source, we need a way to override this when running make.
    • Adding an install target. In this PR, it's configured to be overridden by conventional PREFIX / BINDIR constants. It also supports staged installs using a DESTDIR constant.

    This PR includes some other changes to keep the rest of the Makefile more consistent:

    • Adding an uninstall target, to keep symmetry with install.
    • Defining and using constants for GO and other executables.
    • Adding a new default all target that builds cog as a prerequisite.
    • Adding go clean to the clean target.

    Finally, I noticed a TODO comment from @bfirsh for the test-integration target:

    TODO(bfirsh): use local copy of cog so we don't have to install globally

    With the new cog target, we can run integration tests against local executables without installing them by prepending PWD to PATH. (afbd925bdb2458291493dc6b939d37b9a7d018e5)

    opened by mattt 9
  • Include timestamps in Redis responses

    Include timestamps in Redis responses

    Timestamps are useful for calculating how long a prediction has taken to run. There are other ways we could get this data (using HTTP streams instead of queueing, and doing accurate timing from the outside), but this is probably the simplest to get started with and arguably the simplest for all sorts of users in the future.

    Signed-off-by: Dominic Baggott [email protected]

    Resolves #589

    opened by evilstreak 8
  • build error

    build error

    when I build a model with cog, I always get this error and fail to build. How can I handle this?

     => ERROR [stage-0  3/12] RUN curl https://pyenv.run | bash &&  git clone https://github.com/momo-lab/pyenv-install-latest.git "$(pyenv root)"/plugin  125.3s
    ------
     > [stage-0  3/12] RUN curl https://pyenv.run | bash &&         git clone https://github.com/momo-lab/pyenv-install-latest.git "$(pyenv root)"/plugins/pyenv-install-latest &&        pyenv install-latest "3.8.2" &&         pyenv global $(pyenv install-latest --print "3.8.2"):
    #8 1.093   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
    #8 1.093                                  Dload  Upload   Total   Spent    Left  Speed
    100   270  100   270    0     0    102      0  0:00:02  0:00:02 --:--:--   102
    #8 3.785 curl: (7) Failed to connect to raw.githubusercontent.com port 443: Connection refused
    #8 3.788 /bin/sh: 1: pyenv: not found
    #8 3.811 Cloning into '/plugins/pyenv-install-latest'...
    #8 124.1 fatal: unable to access 'https://github.com/momo-lab/pyenv-install-latest.git/': gnutls_handshake() failed: The TLS connection was non-properly terminated.
    ------
    executor failed running [/bin/sh -c curl https://pyenv.run | bash &&    git clone https://github.com/momo-lab/pyenv-install-latest.git "$(pyenv root)"/plugins/pyenv-install-latest &&        pyenv install-latest "3.8.2" &&         pyenv global $(pyenv install-latest --print "3.8.2")]: exit code: 128
    ⅹ Failed to build Docker image: exit status 1
    
    
    opened by Hurricane-eye 8
  • Initial working version using Pydantic for type annotations

    Initial working version using Pydantic for type annotations

    The exploration in #343 was sufficiently explored and getting very messy, so I thought I would create a new pull request for the real implementation.

    This is an implementation of #193, #205, and #259 using Pydantic for the type system, FastAPI for the model server, and OpenAPI for the schema.

    Models now look a bit like this:

    from cog import Predictor, Input, Path
    import torch
    
    class ColorizationPredictor(Predictor):
        def setup(self):
            self.model = torch.load("./weights.pth")
    
        def predict(self,
              image: Path = Input(title="Grayscale input image")
        ) -> Path:
            processed_input = preprocess(image)
            output = self.model(processed_input)
            return postprocess(output)
    

    The HTTP model server is a work in progress and will be finalized as part of future work.

    The Redis queue worker remains in place and the API is unchanged. As part of future work we will implement an AMQP API with a better API.

    Commits are significant and this should be reviewed commit by commit. This is an intentionally large and long-running branch -- main is currently used as the documentation for Replicate.

    Todo

    • [x] cog predict with new API
    • [x] Input ordering?
    • [x] Ensure HTTP server is single threaded.
    • [x] HTTP URLs for input and output
    • [x] Add choices option to Input to generate an Enum.
    • [x] cog init template
    • [x] Update getting started docs
    • [x] Update reference docs
    • [x] Rename Predictor to BasePredictor so we can do from cog import BasePredictor, Input, Path without colliding with a Predictor subclass.
    • [x] Better errors if you get output type wrong.
    • [x] ~Reliably set Cog version. In development, the version is set to 0.0.1 which is pretty useless. It should probably be hardcoded somewhere, and the release process needs updating. We need this to be able to switch between old and new Cog API in Replicate.~ We're just going to support version dev everywhere which means "latest".
    • [x] Update examples https://github.com/replicate/cog-examples/pull/8
    • [x] Update the OpenAPI Outputs to be a consistent place. (Simple outputs currently end up in Request object schema)
    • [x] Test returning object as output
    • [x] Test arbitrary JSON objects in output
    • [x] Is go.sum up to date?

    Future (Pydantic + OpenAPI)

    • [ ] Tell users to upgrade if they use @cog.input() with new Cog.
    • [ ] Document nouns to make sure we're all on the same page. "Input"/"Inputs"?
    • [ ] Document how to migrate @input() to Pydantic type annotations.
    • [ ] Go through and fix any TODOs
    • [ ] Make input validation errors work for cog predict in a simple way until we lock down HTTP API
    • [ ] Make input errors work for Redis API & Replicate
    • [ ] Integrate with Replicate - https://github.com/replicate/replicate-web/pull/907
    • [ ] Force users to pick an output type -- throw an error if it's not set. (Couldn't just be Any, but they need to be explicit!)
    • [ ] Clean up temporary files -- both input and output
    • [ ] Do we want to make cog predict with an existing image backwards compatible? ('cos there are old models on Replicate)
    • [ ] What happens if you use pathlib.Path as an input type? Does it work? Should we throw an error? (This might happen if you import wrong package / collide imports)
    • [ ] What happens if you return pathlib.Path from a predict function? Does this work? Should it throw an error?
    • [ ] Should we have a fixed set of supported inputs/outputs somehow?
    • [ ] Do we want to force users to put extra properties in additionalProperties? Should we even allow extra properties to cog.Input()?
    • [ ] Merge examples PR https://github.com/replicate/cog-examples/pull/8

    Far-flung future

    • Vendor Pydantic and FastAPI
    • Add back support for PyTorch and Tensorflow tensors as output. This was removed because it was fiddly to get working with Pydantic. You now need to convert them to numpy arrays.
    • Add back timing information to server.
    • Better error if you forget an output type
      pydantic.error_wrappers.ValidationError: 1 validation error for Prediction
      output
        unexpected value; permitted: None (type=value_error.const; given=hello foo; permitted=(None,))
      
    • Multiple file outputs for cog predict. It currently just outputs the plain JSON if there isn't a single file output.
    • Logging, as described in comment in #343
    • Review request/response objects and make sure we're happy with them.
      • This might be an opportunity to rename the statuses and ensure they're consistent ("success" vs "failed").
      • Do we want the input type to have a second input level so we can have top-level request options?
    • Do we need separate URLs for input/output schema, or can they be reliably fetched from OpenAPI?
    • [ ] Finalize & document HTTP API
    • [ ] Do we want to version the prediction API, or let the client handle it? https://github.com/replicate/cog/issues/94
    • [ ] Test arbitrary objects in input
    • [ ] What if people run new models from a directory of code with old Cog? What happens if we do that? Do we care? #286
    • [ ] What if people run new models from an image with old Cog? Maybe we throw a simple error from /predict?

    Based on (& blocked by)

    • #383
    • #385

    Refs

    Closes #58 Closes #113 Closes #193 Closes #205 Closes #212 Closes #246 Closes #259 Closes #262 Closes #263 Closes #304 Closes #306 Closes #327 Closes #328

    opened by bfirsh 8
  • Throw error if invalid keys exist in cog.yaml

    Throw error if invalid keys exist in cog.yaml

    Fix for:

    Throw error if invalid keys exist in cog.yaml #34

    Approaches:

    1) Use hard-coded key pair to test the yaml file

    Create a list of key pairs or a struct to validate the cog.yaml file

    Pros:

    • Simple & straight forward approach
    • Easy to test and validate it works

    Cons:

    • Hard to implement complex logic
    • Needs more code changes
    • Need to learn go and have knowledge of internal systems to make changes
    • Hard to keep track & validate the different versions of cog.yaml file

    2) Use a JSON Schema to validate

    This approach uses a JSON Schema to validate the cog.yaml file.

    Pros:

    • Uses Industry standard approach to validate the yaml file
    • Most cases will need a simple change to json schmema
    • No need to learn go to understand various fields in cog.yaml
    • The json-schema serves as the documentation.

    Cons:

    • Dependencies on external libraries and validation mechanism
    • Would be hard to understand for people not familiar with jsonSchemas
    • Required editing a large json file to make changes

    Approach Selected: "Using JSONSchema"

    Decided to implement the jsonschema approach based on the pros & cons and especially keeping the end-user in mind and the "open-source" nature of the project.

    The approach will not enforce new rules but only validates that the keys present in the yaml are defined in the schema.

    How to generate the initial JSON schema

    • Create a yaml file containing all the fields present in the cog.yaml
    • Converted the yaml to JSON using this tool: https://onlineyamltools.com/convert-yaml-to-json
    • Then generate the jsonschema using this tool: https://www.jsonschema.net/home

    The schema required few changes:

    • Remove the required field
    • Allow both number and string for python_version field

    Validate command

    The PR also includes a validate command to verify any cog.yaml in the current directory.

    Usage:

    cog validate
    

    Examples:

    Empty cog.yaml

    No error will be thrown and cog will use the default schema generated by the below code:

    func DefaultConfig() *Config {
    	return &Config{
    		Build: &Build{
    			GPU:           false,
    			PythonVersion: "3.8",
    		},
    	}
    }
    
    

    These are all valid cog.yaml files:

    • No File
    • Empty file
    • build:

    Bad yaml file example & errors

    cog.yaml:

    buildd:
    

    Output: ⅹ (root) Additional property builds is not allowed

    cog.yaml:

    buildd:
      python_version: "3.8"
      python_packages:
        - "torch=1.8.0"
    

    Output: ⅹ (root) Additional property buildd is not allowed

    cog.yaml:

    build:
      python_versions: "3.8"
      python_packages:
        - "torch=1.8.0"
    

    Output:

    ⅹ build Additional property python_versions is not allowed ..........................................................................................

    Signed-off-by: Shashank Agarwal 2 [email protected]

    opened by imshashank 8
  • Keeping the homebrew formula up to date

    Keeping the homebrew formula up to date

    brew install cog is a thing now! See https://github.com/replicate/cog/pull/849 and https://github.com/Homebrew/homebrew-core/pull/117929

    It looks like it's pinned at 0,5.1. How should we go about keeping that up to date as we release new versions of Cog? My guess at a couple options:

    • We bump that number whenever we do a release. Does that mean opening a PR on homebrew-core each time? 😬
    • We point it at a tarball off the main branch, then promise ourselves to keep main stable. Then we'd never have to update the formula. Too yolo maybe?
    opened by zeke 7
  • Design & document HTTP API

    Design & document HTTP API

    This is a work-in-progress implementation of #413. I was considering creating an issue to discuss the design of the HTTP API, but I figure a pull request and some documentation might be the most productive medium for doing this.

    The text of the documentation is incomplete and might be partial sentences. Mainly looking here for feedback on the API, then we can complete all the sentences.

    I think it is mostly there, but a few design questions:

    1. Do we like our request and response objects?
    2. Do we like our status names? (Maybe failure instead of failed? Is there some prior art here we can build upon instead of inventing something new?)
    3. #428
    4. Is this compatible with #437 and #443?
    5. File handling is a work in progress and I will create a separate piece of documentation about that.
    6. Is this compatible with a potential future async version?
    7. Do we want to version it? #94
    opened by bfirsh 7
  • Show a replicate.com link after pushing to r8.im

    Show a replicate.com link after pushing to r8.im

    Show a message after a successful push. If the push was to the Replicate registry then also show a link to the model page on replicate.com.

    Screenshot 2022-02-28 at 17 51 48

    (It's using localhost:8100 as the replicate registry in development – in the wild that'd be matching against and replacing r8.im)

    The path not taken

    I talked to @bfirsh about displaying a message from replicate.com rather than hardcoding this into Cog. Unfortunately, there's no existing call made to replicate.com after the docker push command has completed, and there's no way I know of to make docker push display more info by feeding it a suitable response from the server.

    So, although it's much harder to update this when it's in Cog (because rolling out new code changes requires people to update their version), this seems like the right place to have it for now.

    Fixes #210

    opened by evilstreak 7
  • Proposed design for public AMQP queueing API

    Proposed design for public AMQP queueing API

    Context

    We currently have a private queue-based redis API that replicate.com uses to talk to Cog models, and we want to turn that into a well-defined public API that uses standard protocols (in this case AMQP).

    Requirements

    • Run predictions from JSON inputs in a request queue
    • Put JSON output of prediction on a response queue specified in request
    • Read input files from a URL (HTTP/S3/GCS) and write output files to particular location in an S3/GCS bucket
    • Record log output of prediction, in some way that can be read as the prediction is running
    • Put in-progress outputs from the model onto the response queue, as the model is running
    • Validate inputs and outputs based on a JSON Schema
    • Include metrics about run in response: model start up time, request processing time, and system metrics like CPU, RAM, and GPU memory used.
    • Errors and logs in response if prediction fails.
    • Cancel a running prediction.

    Strawman design

    At a high-level, the AMQP API uses the same request and response objects as the HTTP API, defined with an OpenAPI schema. But, instead of HTTP requests and responses, these objects are sent over AMQP request and response queues.

    Requests

    • The message body is the same format as the HTTP API request body being implemented in #378. It contains the input and any other metadata.
    • Message body encoded as JSON (content_type property set to application/json).
    • The response queue is specified with the reply_to property on the message.
    • Files are represented as URIs, prefixed with either file://, http://, https://, s3://, or gs://.
    • Example request body:
    {
      "input": {
        "image": "s3://hooli-input-bucket/hotdog.jpg",
        "disentanglement_threshold": 5
      },
      "output_file_prefix": "s3://hooli-output-bucket/5245/"
    }
    

    Responses

    • The message body is the same format as the HTTP API response body. It contains the status, output, logs, metrics, and other metadata.
    • Files are represented as URIs, like in the response. The model will upload them to output_file_prefix defined in the request. Credentials are provided at runtime via environment variables.
    • The response will be sent to the reply_to queue specified in the request.
    • Message body encoded as JSON (content_type property set to application/json).
    • Includes any message IDs present in the request (message_id, correlation_id properties).
    • Includes timing information and other performance metrics (setup time, prediction time, CPU time, memory usage, etc).
    • Includes logs. For each log message, or some log message updating frequency, this message will be sent multiple times, with status: "processing".
    • For models that return in-progress output as as they are running, this message will be sent multiple times, with status: "processing" and the "output" key set to the output.
    • Example request body:
    {
      "status": "success",
      "output": "s3://hooli-output-bucket/5245/output.jpg",
      "metrics": {
        "setup_time": <time>,
        "prediction_time": <time>
      },
      "logs": <array of strings, one per line>
    }
    

    Schema

    The AMQP API is specified with a subset of the OpenAPI/JSON Schema used for the HTTP API.

    It is available both as a org.cogmodel.openapi_schema label on the image, and at /openapi.json in the HTTP API.

    The relevant subset looks like this:

    {
      "components": {
        "schemas": {
          "Input": {
            "title": "Input",
            "required": ["image", "disentanglement_threshold"],
            "type": "object",
            "properties": {
              "image": {
                "description": "Image to manipulate",
                "type": "string",
                "format": "uri",
                "x-order": 0
              },
              "disentanglement_threshold": {
                "type": "number",
                "x-order": 1
              }
            }
          },
          "Output": {
            "title": "Output",
            "type": "string",
            "format": "uri"
          }
        }
      }
    }
    
    

    Big differences between old and new API

    • New system would use a well-defined, public AMQP API instead of the current system of using redis queues, to run predictions. Rationale: want to use standard tools, and make it and easy to run predictions in other workflows.
    • Old system had separate queues for responses, logs, and timing info, new system puts all of that info in the response queue. Rationale: easier to keep track of 1 queue?

    Todo

    • Cancel running predictions
    • Error handling
    • Pass credentials for file downloads/uploads with a request for scoping
    • Backwards compatibility on Replicate

    Some references worth reading:

    Edited by @preeth1, @bfirsh, ... (maintainers -- add your name here! consider this is a wiki.)

    opened by preeth1 7
  • Bump github.com/mattn/go-isatty from 0.0.16 to 0.0.17

    Bump github.com/mattn/go-isatty from 0.0.16 to 0.0.17

    Bumps github.com/mattn/go-isatty from 0.0.16 to 0.0.17.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies go 
    opened by dependabot[bot] 0
  • Bump github.com/getkin/kin-openapi from 0.110.0 to 0.112.0

    Bump github.com/getkin/kin-openapi from 0.110.0 to 0.112.0

    Bumps github.com/getkin/kin-openapi from 0.110.0 to 0.112.0.

    Release notes

    Sourced from github.com/getkin/kin-openapi's releases.

    v0.112.0

    What's Changed

    New Contributors

    Full Changelog: https://github.com/getkin/kin-openapi/compare/v0.111.0...v0.112.0

    v0.111.0

    What's Changed

    New Contributors

    Full Changelog: https://github.com/getkin/kin-openapi/compare/v0.110.0...v0.111.0

    Commits
    • 46e0df8 openapi3filter: use option to skip setting defaults on validation (#708)
    • a0b67a0 openapi3: continue validation on valid oneOf properties (#721)
    • 1f680b5 feat: improve error reporting for bad/missing discriminator (#718)
    • 1490eae openapi3: introduce (Paths).InMatchingOrder() paths iterator (#719)
    • de2455e openapi3: unexport ValidationOptions fields and add some more (#717)
    • 3be535f openapi3filter: validate non-string headers (#712)
    • 25a5fe4 Leave allocation capacity guessing to the runtime (#716)
    • 2975a21 openapi3: patch YAML serialization of dates (#698)
    • 35bb627 Fix links to OpenAPI spec after GitHub changes (#714)
    • 6a3b779 Fix inconsistent processing of server variables in gorillamux router (#705)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies go 
    opened by dependabot[bot] 0
  • Allow redis connection URL to connect to redis.

    Allow redis connection URL to connect to redis.

    Related to #747

    This is implementing --redis-url CLI option to connect to redis as described in the comments of #747

    It's useful because allowing a URL as an alternative to just host/port allows for more options when using redis

    • Using SSL to connect
    • Using username/password to connect
    • Using a different database

    Existing --redis-host and --redis-port adds a deprecation warning but still works as before.

    opened by bbedward 1
  • Is Redis worker supposed to exit after completing a single job?

    Is Redis worker supposed to exit after completing a single job?

    I've been trying to make Redis worker work with no success, unless I'm misunderstanding something. I'm using this example with my own cog image: https://github.com/replicate/cog-redis-example

    I add a message to the queue, the worker picks it up. Logs shows that it processes it but then it exits with the message:

    Shutting down worker: bye bye!

    No error messages, just that. I'm wondering if this is the intended behaviour of a cog. I expected it to continue listening for messages but maybe I understood the worker wrong. If this is the intended behaviour, are there any ways to make it not exit after a single job completion?

    opened by yekta 1
  • Bump github.com/docker/docker from 20.10.21+incompatible to 20.10.22+incompatible

    Bump github.com/docker/docker from 20.10.21+incompatible to 20.10.22+incompatible

    Bumps github.com/docker/docker from 20.10.21+incompatible to 20.10.22+incompatible.

    Release notes

    Sourced from github.com/docker/docker's releases.

    v20.10.22

    Bug fixes and enhancements

    • Improve error message when attempting to pull an unsupported image format or OCI artifact (moby/moby#44413, moby/moby#44569).
    • Fix an issue where the host's ephemeral port-range was ignored when selecting random ports for containers (moby/moby#44476).
    • Fix ssh: parse error in message type 27 errors during docker build on hosts using OpenSSH 8.9 or above (moby/moby#3862).
    • seccomp: block socket calls to AF_VSOCK in default profile (moby/moby#44564).

    Packaging Updates

    Commits
    • 42c8b31 Merge pull request #44656 from thaJeztah/20.10_containerd_binary_1.6.13
    • ff29c40 update containerd binary to v1.6.13
    • 0234322 Merge pull request #44488 from thaJeztah/20.10_backport_update_gotestsum
    • edca413 [20.10] update gotestsum to v1.8.2
    • 6112b23 Merge pull request #44476 from sbuckfelder/20.10_UPDATE
    • 194e73f Merge pull request #44607 from thaJeztah/20.10_containerd_binary_1.6.12
    • a9fdcd5 [20.10] update containerd binary to v1.6.12 (addresses CVE-2022-23471)
    • 48f955d Merge pull request #44597 from thaJeztah/20.10_containerd_1.6.11
    • 50d4d98 Merge pull request #44569 from thaJeztah/20.10_backport_relax_checkSupportedM...
    • 17451d2 Merge pull request #44593 from thaJeztah/20.10_update_go_1.18.9
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies go 
    opened by dependabot[bot] 0
  • Bump github.com/docker/cli from 20.10.21+incompatible to 20.10.22+incompatible

    Bump github.com/docker/cli from 20.10.21+incompatible to 20.10.22+incompatible

    Bumps github.com/docker/cli from 20.10.21+incompatible to 20.10.22+incompatible.

    Commits
    • 3a2c30b Merge pull request #3919 from thaJeztah/20.10_update_engine
    • 47649fb vendor: github.com/docker/docker v20.10.21
    • 3b562e9 vendor: github.com/moby/buildkit v0.8.4-0.20221020190723-eeb7b65ab7d6
    • e7cdabe Merge pull request #3918 from thaJeztah/20.10_docs_backports
    • 5106d8e Merge pull request #3917 from thaJeztah/20.10_backport_update_gotestsum
    • ce10682 Remove deprecated note
    • 058f7df docs: docker inspect --size
    • 226a2fd docs: docker inspect: reformat with prettier
    • 42eca75 docs: use correct separator in --security-opt
    • 0c8ce43 docs: fix misleading example of setting an env variable for a single command
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies go 
    opened by dependabot[bot] 0
Releases(v0.6.1)
  • v0.6.1(Dec 15, 2022)

  • v0.6.0(Dec 15, 2022)

    Welcome to Cog v0.6.0! Three changes this time:

    • Configure a new docker entrypoint. This makes sure the container respects SIGTERM. It also fixes an issue caused by Nvidia changing entrypoints in all CUDA base images. https://github.com/replicate/cog/pull/844
    • Expose the version of Cog within the Python package, via cog.__version__. https://github.com/replicate/cog/pull/843
    • Update the HTTP API to use the new Worker class introduced in v0.5.0. This is the first step towards an asynchronous API that matches Replicate's API. https://github.com/replicate/cog/pull/824

    Full changelog

    • 0bb3ace Add a busy test for PredictionRunner
    • 79851e8 Add tests for cog.server.runner.predict()
    • d2726e0 Configure Cog python package version from git
    • 7f49116 Extract predictor fixtures for Cog HTTP tests
    • 5303d8a Install tini as image entrypoint
    • a9569db Lookup the schema for the predictions API by path
    • 04c4215 Make File inputs survive pickling/unpickling
    • 8b67617 Use Worker for Cog HTTP API and respond like Replicate
    • bbe999e document brew install cog
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.58 MB)
    cog_Darwin_x86_64(9.84 MB)
    cog_Linux_arm64(8.68 MB)
    cog_Linux_x86_64(9.10 MB)
  • v0.5.1(Dec 8, 2022)

    Changelog

    • bf08e07 Add an explicit handler to die on SIGTERM
    • 1e13500 Add anchor for removed Update section
    • 25ba72d Add cog target
    • d934417 Add console language identifier to code blocks
    • f447eec Add default all target
    • 26aa17f Add instructions for building from source
    • b161aca Add uninstall target
    • 3e5d555 Bump golang.org/x/sys from 0.2.0 to 0.3.0
    • e4131de Bump golang.org/x/tools from 0.3.0 to 0.4.0
    • fa87b16 Consolidate instructions for updating
    • d949133 Copy editing Install section
    • 8cc4fe9 Define constants for go and python executables
    • 3cc673a Don't do Python stuff when testing Go
    • 3bb2169 Don't pin dependencies that are unpinned in setup.py
    • d83f75b Fix CI
    • 0ded3c3 Ignore built cog executable
    • f2f5199 Make cog HTTP server port configurable
    • 90c4ce4 Only ignore the built cog binary, not anything named cog :)
    • a94662e Read go version from go.mod in CI workflow definition
    • fd5f712 Remove dollar sign prefix in pastable commands
    • 3800f83 Remove instruction to download Docker from Install section
    • d83584d Set LDFLAGS in cog target
    • f7a2cd8 Specify requirements for integration tests
    • 6b68d03 Support staged installs
    • 0adf04e Update install target
    • 8685a32 Update tests and FastAPI pin for schema changes
    • 2ed0360 Upgrade to Go 1.19
    • 0d7bbad Use built executables for integration tests
    • 2fdeefa Use dollar sign for command substitution
    • b58bd63 empty commit to trigger CI when next tag is created
    • 1772f69 update .all-contributorsrc [skip ci]
    • 9593f7c update .all-contributorsrc [skip ci]
    • 9f6ebcd update .all-contributorsrc [skip ci]
    • fa6aeb8 update .all-contributorsrc [skip ci]
    • 6a6d5c0 update .all-contributorsrc [skip ci]
    • 3ccea04 update .all-contributorsrc [skip ci]
    • 340d8fb update .all-contributorsrc [skip ci]
    • b94f0df update .all-contributorsrc [skip ci]
    • 268dec0 update README.md [skip ci]
    • ba10aa5 update README.md [skip ci]
    • af3cf46 update README.md [skip ci]
    • 6572830 update README.md [skip ci]
    • febe58f update README.md [skip ci]
    • 9b35dcf update README.md [skip ci]
    • 35110c9 update README.md [skip ci]
    • c275af2 update README.md [skip ci]
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.56 MB)
    cog_Darwin_x86_64(9.84 MB)
    cog_Linux_arm64(8.68 MB)
    cog_Linux_x86_64(9.10 MB)
  • v0.5.0(Dec 1, 2022)

    Welcome to Cog v0.5.0! It's been a while since we've made a release, and there's a lot of stuff going on here.

    If you're using the Redis queue worker, this is a breaking release. If you're not using that worker, this release is backwards compatible with previous versions of Cog.

    Almost all of this release is about the Redis queue worker. We've been rebuilding the infrastructure for Replicate – improving the reliability and performance – and in the process have been moving some of the features of our serving platform directly into Cog. If you're using Cog to run your own models in production, then these changes are for you.

    We're not done yet though! We're currently working on the HTTP interface, bringing it up to scratch. And we've got some big plans to use that updated interface to make serving machine learning models even easier and safer.

    Changes

    First up, a couple of quality of life fixes:

    • Stop container when ctrl+c is pressed during cog predict #751
    • Fix builds with buildx #798

    And now, the Redis queue worker:

    • can send responses back as HTTP webhook calls, instead of via Redis (#727)
    • can filter the events it'll send webhooks for (#807)
    • has an updated response format, matching the API for Replicate (#729 & #743)
    • makes file uploads as a basic PUT request, so it can upload directly to cloud storage, without needing special endpoints (#735)
    • can be configured to exit after too many failures, to protect against workers getting into a bad state (#800)
    • sends status, timings and logs for setup to a URL, so you can monitor performance and react to failures (#803)
    • writes a file when setup is complete, so you can create readiness checks via tools like Kubernetes (#805)
    • can be started with keyword arguments, instead of an increasingly long list of positional arguments (#775)
    • can handle cancelations internally, by giving it a Redis key to watch for changes (#746)
    • has a completely rewritten core, to improve the reliability and prepare us for updating the HTTP server to support most of the features above (#795)

    Full changelog

    • 0c3466c Accept list of events to send webhooks for
    • 93367e4 Add CUDA 11.8 and Tensorflow 2.11.0
    • 01e9f7f Add metrics.predict_time to response output
    • a3b97e6 Add option to exit redis worker after too many failures
    • 1e5435d Add support for --extra-index-url to PyTorch
    • fe5696d Add tests for StreamRedirector, and make line buffering consistent
    • d74c142 Add the ability to suppress teeing of subprocess output
    • 5cf6d0a Add unit tests for WrappedStream
    • a3ca53a Add user-agent to webhook calls
    • e44593c Add write()/flush() to WrappedStream
    • ae2cb44 Address timing issue with cancel signal
    • 1f8fec1 Bodge in support for Python 3.11
    • c8058fb Bump github.com/docker/cli
    • e3df72f Bump github.com/docker/docker
    • 1683b8b Bump github.com/getkin/kin-openapi from 0.109.0 to 0.110.0
    • 72c2839 Bump github.com/getkin/kin-openapi from 0.98.0 to 0.109.0
    • 6799530 Bump github.com/golangci/golangci-lint from 1.49.0 to 1.50.0
    • 909a184 Bump github.com/golangci/golangci-lint from 1.50.0 to 1.50.1
    • 3fa833d Bump github.com/spf13/cobra from 1.5.0 to 1.6.1
    • 5c489f3 Bump golang.org/x/tools from 0.1.12 to 0.3.0
    • 338db31 Bump pillow from 9.0.1 to 9.3.0
    • cef47fa Caller is responsible for wrapping streams
    • 436b12d Canceled is a terminal status
    • 5ad419f Echo back request fields in queue worker responses
    • 8470901 Extract Dockerfile temporary file logic
    • 7afd940 Extract format_datetime method for queue worker
    • 017ac89 Fetch latest PyTorch version from pip repository
    • 3d43593 Fix a small logic bug
    • 7003fec Fix deadlock for large outputs
    • 12137aa Fix past PyTorch version parsing
    • 8c4e84a Fix tests
    • 51ed797 Fix typo
    • 2314dc2 Flush tee-d output after each line
    • a55ad5a Handle cancelations in Redis queue worker
    • c2c8c83 Handle dead predictor process
    • e151d1d Install Python requirements from file
    • 8f1f419 Log cancel signals to stderr
    • 4782420 Loosen attrs dependency as far as possible
    • 523b8f1 Make "Status" a string forward ref to avoid test errors
    • f511232 Make a start on a more realistic state-based fuzz test
    • 7230dc8 Make optional types explicit
    • d6fc5d6 Minor setup error handling tweaks for clarity
    • e06bc35 Move handling of Done into _predict
    • 415918a Move x-experimental-timestamps to response root
    • 2587fc1 Mypy fixes
    • 1676cf4 New Worker implementation (replaces Runner)
    • 1ac5b92 Pass response_interval to ResponseThrottler
    • 987ebb8 Pass through lines in requirements.txt that aren't pinned
    • 508eabc Put pip --find-links in requirements.txt instead of inline
    • 17ee469 Read python_requirements and resolve versions from it
    • a2df984 Refactor redis_queue to extract response building
    • d3b0592 Remove python_extra_index_urls and python_find_links
    • 248a733 Rename IndexURL to correct "FindLinks"
    • 2e596a2 Rename PythonPackagesForArch to PythonRequirementsForArch
    • 4890fac Rename ValidateAndCompleteConfig to ValidateAndComplete
    • b516768 Require protobuf<=3.20.3
    • e822bc8 Retry webhook for terminal responses
    • 8ceb8fd Return a string from PythonRequirementsForArch
    • df3b04c Satisfy the mypy gods
    • 2fe5872 Send file uploads as a basic PUT
    • cf88ed4 Send logs lines as newline-separated string instead of list
    • 7caa858 Send timings and logs for setup to a URL
    • 8798eea Set HYPOTHESIS_PROFILE for GitHub Actions
    • 74b05e0 Set default throttle interval to 500ms
    • 936afd7 Shut down if the runner died
    • 4b54bab Switch to keyword arguments for queue worker
    • ec63f96 Throttle webhook responses
    • ce41c84 Update python.md
    • 2d80011 Update yaml.md
    • 01600e1 Use ValidateAndCompleteConfig in config tests
    • 0ed776c Use future annotations instead of string type hint
    • 4adcb5d Work around STDIO buffering in tests
    • efbe725 Write a file when setup is complete
    • 2483538 Yet more protection around cancelation!
    • 002c14e black/isort run
    • 745f003 gitignore .DS_Store
    • 469d36d load flag is required in buildx
    • 31872fd python-openssl system package not used
    • e5dafee rename installpython function to include scope - cuda
    • 76e9289 stop container on ctrl-c during cog predict (#751)
    • 63ca65d update .all-contributorsrc [skip ci]
    • 78f61e7 update README.md [skip ci]
    • 163eef7 update docs to 1.18 version of Go (used in CI)
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.42 MB)
    cog_Darwin_x86_64(9.62 MB)
    cog_Linux_arm64(8.62 MB)
    cog_Linux_x86_64(8.91 MB)
  • v0.4.4(Sep 23, 2022)

  • v0.4.3(Sep 20, 2022)

  • v0.4.2(Aug 23, 2022)

    Changelog

    • 8504420 Bump github.com/golangci/golangci-lint from 1.47.3 to 1.48.0
    • 0d6b2b1 Bump github.com/mattn/go-isatty from 0.0.14 to 0.0.16
    • b91e868 Bump gotest.tools/gotestsum from 1.8.1 to 1.8.2
    • 751c0cb Fix custom output file names
    • e03fb84 Output multiple files
    • 3c59cf1 Suggest upgrading Cog if CUDA version can't be found
    • 427530d document returning a list
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.32 MB)
    cog_Darwin_x86_64(9.53 MB)
    cog_Linux_arm64(8.43 MB)
    cog_Linux_x86_64(8.83 MB)
  • v0.4.1(Aug 4, 2022)

    Changes

    • You can now use unsupported Tensorflow versions with GPUs. If you set a Tensorflow version that isn't supported, you also need to set the cuda option to choose a compatible CUDA version.

    Full changelog

    • eefcbc6 Allow unsupported tensorflow versions
    • a3bce61 Bump github.com/getkin/kin-openapi from 0.97.0 to 0.98.0
    • 227ddc1 Bump github.com/golangci/golangci-lint from 1.47.1 to 1.47.3
    • 82d913c Deterministically select base images
    • ef22847 Return error if CUDA version is not supported
    • 089f9e9 User Guide - Run cog on Windows 11 using WSL 2 (#681)
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.32 MB)
    cog_Darwin_x86_64(9.53 MB)
    cog_Linux_arm64(8.43 MB)
    cog_Linux_x86_64(8.82 MB)
  • v0.4.0(Jul 27, 2022)

    Changes

    • Breaking change: the Redis queue worker now uses SET instead of RPUSH to send responses. #684
    • Improve behaviour of queue worker when reaching a maximum prediction length timeout #679

    Full changelog

    • b3cd0fe Bump github.com/golangci/golangci-lint from 1.46.2 to 1.47.1
    • d6e8156 Bump github.com/stretchr/testify from 1.7.5 to 1.8.0
    • eb6b53e Bump github.com/xeonx/timeago from 1.0.0-rc4 to 1.0.0-rc5
    • 1615005 Fix the otel span for predictor.predict
    • e9d6c2c Log traceback on predict() error
    • 436c254 Move timeout from queue worker into runner
    • 3a22b07 Rename max_processing_time -> autoclaim_messages_after
    • 38d94ca Update macos test version from 10.15 to 12
    • 6c64e12 Use a string key for responses instead of a list
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.32 MB)
    cog_Darwin_x86_64(9.52 MB)
    cog_Linux_arm64(8.43 MB)
    cog_Linux_x86_64(8.82 MB)
  • v0.3.11(Jun 28, 2022)

    Changes

    A breaking change in redis-py 4.3.4 breaks Cog's queue worker. This release handles the new XAUTOCLAIM response format in a backwards-compatible way.

    If you're upgrading Cog to this version to get a working queue worker, you just need to rebuild and push after upgrading. No changes to your code are necessary.

    Changelog

    • 91a2745 Bump github.com/stretchr/testify from 1.7.4 to 1.7.5
    • 9b6bb20 Handle new XAUTOCLAIM response introduced in redis>=4.3.4
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.32 MB)
    cog_Darwin_x86_64(9.52 MB)
    cog_Linux_arm64(8.43 MB)
    cog_Linux_x86_64(8.82 MB)
  • v0.3.10(Jun 22, 2022)

    Changes

    A bumper release, with five improvements to the queue worker:

    • We've added some OpenTelemetry instrumentation for distributed tracing #646
    • Start and finish timestamps are includes with responses, in an experimental property #599
    • The worker handles a SIGTERM better, shutting down more quickly #658
    • A response is sent when the worker starts a prediction, for quicker feedback #657
    • A fix to support output like Iterator[List[Output]] #653

    Full changelog

    • 3787884 Add HTTP test for iterator of list of complex output
    • a6f002c Add OpenTelemetry instrumentation to queue worker
    • ce434ce Add docs for queue worker telemetry
    • f0d45d8 Bump github.com/spf13/cobra from 1.4.0 to 1.5.0
    • df737d5 Bump github.com/stretchr/testify from 1.7.2 to 1.7.4
    • 9848edf Fix bad variable in log line
    • 3b1553f Handle SIGTERM gracefully
    • 61c5fc4 Ignore OpenTelemetry typing issues
    • da5f485 Include timestamps in Redis responses
    • b3bec59 Pin protobuf to make opentelemetry work
    • cbfe5bf Record time in queue in opentelemetry span
    • ba12047 Remove unused variable: start_time
    • c104704 Send response when queue worker starts predict
    • fcbe521 Split encode_json() into make_encodeable() and upload_files()
    • fa3a472 Use blpop not brpop in Redis tests
    • 9a20b26 Use make_encodeable inside prediction runner
    • 01bd1bb document how to optionally set output properties
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.32 MB)
    cog_Darwin_x86_64(9.52 MB)
    cog_Linux_arm64(8.43 MB)
    cog_Linux_x86_64(8.82 MB)
  • v0.3.9(Jun 14, 2022)

  • v0.3.8(Jun 8, 2022)

  • v0.3.6(Jun 7, 2022)

  • v0.3.5(Jun 1, 2022)

  • v0.3.4(May 19, 2022)

    Changes

    • When passing the -i option to cog predict without the name of an input, it now defaults to the first. Previously, it would default to magic input called input.

    Full changelog

    • deb4c04 Add mypy to vscode and lint
    • 8fd6c77 Appease mypy gods
    • f1ed980 Bump github.com/docker/cli
    • 8b3a613 Bump github.com/docker/cli
    • 585f8eb Bump github.com/docker/docker
    • 9c0b86a Bump github.com/docker/docker
    • 2712640 Bump github.com/golangci/golangci-lint from 1.45.2 to 1.46.2
    • ab35949 Bump gotest.tools/gotestsum from 1.8.0 to 1.8.1
    • 57354dd Fix type for choices
    • 04726d5 Fixes #592
    • 82a569c Make default input first input
    • 96d1a8f Require typing-extensions>=4.1.0
    • 25cfd50 Update README.md
    • 1323692 Update pkg/cli/init-templates/predict.py
    • 765b4a1 Update remote image test to recent Cog model
    • 498fe29 Update test-integration/test_integration/test_predict.py
    • 747b513 Updates formatting in getting-started, drops --gpus
    • 92a70b1 Upgrade to Go 1.18
    • 2a725db go mod tidy
    • 159cfc7 only catch AttributeError
    • 7e96884 return str(t) if there's no qualname in t
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(8.97 MB)
    cog_Darwin_x86_64(9.15 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.47 MB)
  • v0.3.3(May 3, 2022)

  • v0.3.2(Apr 30, 2022)

  • v0.3.1(Apr 27, 2022)

  • v0.3.0(Apr 20, 2022)

    Hightlights

    Cog now has an experimental ⚡️ built-in queue worker ⚡️ that can process predictions from a Redis queue. See docs/redis.md.

    This release also includes lots of bugfixes and improvements to the documentation.

    Breaking changes

    • 311ffc0 The success status has been change to succeeded in the HTTP API and throughout Cog

    Changelog

    • 1cde0ae Add experimental notice to queue API
    • 3e1447b Add logs to queueing API responses
    • e010d1e Allow version to be specified with make install
    • 5d82ba0 Document that file handling is subject to change
    • 0f7128f Document the Redis queue API
    • 1ef7bbc Don't expect prediction ID from queueing API request
    • 5e265ab Fix curl command and add real image
    • 0defd8b Fix enums being passed to predict() when using choices
    • bacaf22 Fix getting started guide
    • 290d987 Include output and logs with failed responses
    • 90c78e7 Remove input wrapping for queueing API
    • af91725 Remove redis_log_capture
    • 9b69a58 Rename inputs to input for queueing API requests
    • 4f2271c Rename output key from "value" to "output"
    • 311ffc0 Rename success status to succeeded
    • 55875b0 Rework predictor subprocess to be long running
    • 4d3bff2 Support complex outputs from predictors
    • 5e1824e Use an Enum for whether output is a generator
    • 5c7201b fix notebook docs
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.25 MB)
    cog_Darwin_x86_64(9.45 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.78 MB)
  • v0.2.1(Apr 15, 2022)

  • v0.2.0(Apr 4, 2022)

    Changelog

    • 32865a1 Add EXPOSE to Dockerfile (#517)
    • f1ca5cc Add Python 3.10 support for PyTorch
    • 4591e1c Add documentation to explain how to deploy models (#526)
    • e05b0c1 Bump github.com/docker/docker
    • bb64680 Bump github.com/getkin/kin-openapi from 0.92.0 to 0.93.0 (#512)
    • da71b5d Bump github.com/getkin/kin-openapi from 0.93.0 to 0.94.0
    • 9ad0a58 Define some nouns
    • e7abca8 Document HTTP API
    • 3e1e608 Fetch latest torch versions from PyPi
    • d1426c0 Fix output documentation
    • 45f351a Install pyenv from github (#525)
    • 6724526 Make it clear what is incomplete (#528)
    • 26c82a0 More forgiving parser to deal with --extra-index-url
    • d73af5c Pick latest patch version for CUDA
    • 0528f3c Relax generate compatibility to handle ROCm
    • dadab64 Rename run.cog.cog_version label to run.cog.version (#532)
    • e855d3d Revert "Update compatibility matrices (#504)" (#530)
    • cb707f7 Switch to cog.run labels
    • b057e19 Update compatibility matrices
    • 488203e Update compatibility matrices (#504)
    • dd4d325 add -p flag tocog run to publish ports
    • 9c8d6a2 document how to use notebooks with cog (#524)
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.25 MB)
    cog_Darwin_x86_64(9.45 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.78 MB)
  • v0.1.4(Mar 25, 2022)

    Changelog

    • dd9e228 Add Schema test for int enums
    • 50c0295 Bump github.com/docker/cli (#510)
    • 20b3a0e Bump github.com/golangci/golangci-lint from 1.44.2 to 1.45.0 (#501)
    • cad7cfb Bump github.com/golangci/golangci-lint from 1.45.0 to 1.45.2 (#509)
    • 9c7c4a9 Bump github.com/stretchr/testify from 1.7.0 to 1.7.1
    • 556dcfc Bump golang.org/x/tools from 0.1.9 to 0.1.10 (#492)
    • 709c807 Fix int enums
    • 5dc27a2 Update README (#507)
    • e08392e document input and output types (#503)
    • 244a811 fix invalid empty lists from cog init (#493)
    • 7464b27 make heredocs consistent for Request and Response
    • 451882a remove stops from heredocs for consistency
    • 4505b80 s/build, ship, run/package/ (#508)
    • 9c26117 use ge and le options in predictor template
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.23 MB)
    cog_Darwin_x86_64(9.44 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.76 MB)
  • v0.1.3(Mar 15, 2022)

    Changelog

    • 12707a1 Bump github.com/docker/cli
    • eeff3cd Bump github.com/docker/docker
    • afe6826 Bump github.com/getkin/kin-openapi from 0.91.0 to 0.92.0 (#487)
    • e1d5729 Bump github.com/spf13/cobra from 1.3.0 to 1.4.0
    • 34ae4d6 Fix yielding predictor cleanup bug in HTTP server
    • 3f7bfa6 Remove lies (#477)
    • 814f002 Support int choices
    • d39709c document cog debug dockerfile (#474)
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.23 MB)
    cog_Darwin_x86_64(9.44 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.76 MB)
  • v0.1.2(Mar 9, 2022)

    A little release that fixes the choices option to Input and progressive output when there are input files.

    Changelog

    • 7464588 Add redis queue test for inputting and yielding files
    • bc3ea9d Add type to Response.status schema
    • 5184cab Add types to enums in the schema
    • 6c6988d Bump github.com/getkin/kin-openapi from 0.90.0 to 0.91.0
    • 5dec7f2 Fix yielding predictor cleanup bug
    • 4898008 Remove cog.Predictor
    • 5d6197f document how to yield progressive output
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.23 MB)
    cog_Darwin_x86_64(9.44 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.76 MB)
  • v0.1.1(Mar 7, 2022)

    This small release includes support for progressive output. (docs forthcoming).

    👉🏼 If you're upgrading from 0.0.x, see the 0.1.0 release notes for details on how to upgrade your model to be compatible with 0.1.x versions.

    Changelog

    • e5497b8 Fix progressive output being one iteration behind
    • aa30ec7 Make progressive output a list
    • cf94f63 Show a replicate.com link after pushing to r8.im
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.23 MB)
    cog_Darwin_x86_64(9.43 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.76 MB)
  • v0.1.0(Mar 2, 2022)

    👷‍♀️ This new version of Cog includes significant changes from previous versions. If you encounter any problems using it, please open an issue and we'll help you out! :octocat:

    What's changed?

    It's all about the schema! {...}

    Cog's Python API has been updated to use the popular Pydantic library under the hood. Pydantic makes use of Python's type annotations (a native feature of the Python language) to help validate your code and prevent runtime errors. But perhaps most importantly, Pydantic can convert a Python file into a JSON schema.

    Cog now generates a JSON schema from the Predictor you've defined in your predict.py file, and uses another popular Python package called FastAPI to dynamically build an OpenAPI schema and HTTP web server that can be used as a JSON API to generate predictions from your model. And because it's using OpenAPI, this server also dynamically generates documentation specific to your model, and a Swagger UI interface for trying out the API.

    With this new schema-focused design, the inputs and outputs you define for your model are converted into a portable, language-agnostic structured data format. This helps guide us toward a more standardized way of defining model interfaces and paves the way to better documentation, more automated tests, easier integrations across systems, and less reinventing the wheel.

    Upgrading

    First, install Docker if you haven't already. Then run this in a terminal:

    sudo rm $(which cog) # (if you have a previous version of cog installed)
    sudo curl -o /usr/local/bin/cog -L https://github.com/replicate/cog/releases/latest/download/cog_`uname -s`_`uname -m`
    sudo chmod +x /usr/local/bin/cog
    

    Examples

    To get a feel for the changes, check out the replicate/cog-examples repository which has been updated with examples supporting this new version:

    • https://github.com/replicate/cog-examples/blob/main/blur/predict.py
    • https://github.com/replicate/cog-examples/blob/main/hello-world/predict.py
    • https://github.com/replicate/cog-examples/blob/main/resnet/predict.py

    Upgrading from Cog 0.0.x to 0.1.x

    Reference docs for Cog's new Python API can be found here.

    In Cog versions up to 0.0.20, you described inputs using @cog.input decorators on your predict method. For example:

    @cog.input("image", type=Path, help="Image to enlarge")
    @cog.input("scale", type=float, default=1.5, help="Factor to scale image by")
    def predict(self, image, scale):
        ...
    

    From Cog 0.1.0 onwards, we've started using Pydantic to define input and output types. Rather than describing inputs using decorator annotations, you now describe them with Python's built-in type hinting. Here's how you'd define the same inputs now:

    def predict(self,
        image: Path = Input(description="Image to enlarge"),
        scale: float = Input(description="Factor to scale image by", default=1.5)
    ) -> Path:
        ...
    

    The parameters that Input() takes are pretty similar to those @cog.input() used to take. Here are the differences:

    • It no longer takes a type parameter; use a type hint instead.
    • The help parameter has been renamed to description.
    • The options parameter has been renamed to choices.
    • The min option has been replaced with ge (greater than or equal to)
    • The max option has been replaced with le (less than or equal to)

    The other major difference is that you now need to define the output type of your model. That's the -> Path bit in the example above. That might be a simple type like str, float or bool. If you need to handle multiple outputs, check out the new documentation for complex output objects.

    If you have any trouble using this new version of Cog, please open an issue. :octocat:

    Feedback

    This new version of Cog includes a number of internal and user-facing changes from previous versions, so there could be some rough edges.

    If you encounter any problems using it, please open an issue and we'll help you out! :octocat:

    You can also chat with us on Discord at discord.gg/replicate. 💬

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.23 MB)
    cog_Darwin_x86_64(9.43 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.76 MB)
Owner
Replicate
Making machine learning reproducible
Replicate
Machine Learning for Go

GoLearn GoLearn is a 'batteries included' machine learning library for Go. Simplicity, paired with customisability, is the goal. We are in active deve

Stephen Whitworth 8.7k Jan 3, 2023
On-line Machine Learning in Go (and so much more)

goml Golang Machine Learning, On The Wire goml is a machine learning library written entirely in Golang which lets the average developer include machi

Conner DiPaolo 1.4k Jan 5, 2023
Gorgonia is a library that helps facilitate machine learning in Go.

Gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily

Gorgonia 4.8k Dec 30, 2022
Machine Learning libraries for Go Lang - Linear regression, Logistic regression, etc.

package ml - Machine Learning Libraries ###import "github.com/alonsovidales/go_ml" Package ml provides some implementations of usefull machine learnin

Alonso Vidales 196 Nov 10, 2022
Gorgonia is a library that helps facilitate machine learning in Go.

Gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily

Gorgonia 4.8k Dec 27, 2022
Prophecis is a one-stop machine learning platform developed by WeBank

Prophecis is a one-stop machine learning platform developed by WeBank. It integrates multiple open-source machine learning frameworks, has the multi tenant management capability of machine learning compute cluster, and provides full stack container deployment and management services for production environment.

WeBankFinTech 392 Dec 28, 2022
Go Machine Learning Benchmarks

Benchmarks of machine learning inference for Go

Nikolay Dubina 25 Dec 30, 2022
A High-level Machine Learning Library for Go

Overview Goro is a high-level machine learning library for Go built on Gorgonia. It aims to have the same feel as Keras. Usage import ( . "github.

AUNUM 351 Nov 20, 2022
Katib is a Kubernetes-native project for automated machine learning (AutoML).

Katib is a Kubernetes-native project for automated machine learning (AutoML). Katib supports Hyperparameter Tuning, Early Stopping and Neural Architec

Kubeflow 1.3k Jan 2, 2023
PaddleDTX is a solution that focused on distributed machine learning technology based on decentralized storage.

中文 | English PaddleDTX PaddleDTX is a solution that focused on distributed machine learning technology based on decentralized storage. It solves the d

null 82 Dec 14, 2022
Self-contained Machine Learning and Natural Language Processing library in Go

Self-contained Machine Learning and Natural Language Processing library in Go

NLP Odyssey 1.3k Jan 8, 2023
Example of Neural Network models of social and personality psychology phenomena

SocialNN Example of Neural Network models of social and personality psychology phenomena This repository gathers a collection of neural network models

null 7 Dec 5, 2022
Reinforcement Learning in Go

Overview Gold is a reinforcement learning library for Go. It provides a set of agents that can be used to solve challenges in various environments. Th

AUNUM 306 Dec 11, 2022
Spice.ai is an open source, portable runtime for training and using deep learning on time series data.

Spice.ai Spice.ai is an open source, portable runtime for training and using deep learning on time series data. ⚠️ DEVELOPER PREVIEW ONLY Spice.ai is

Spice.ai 774 Dec 15, 2022
FlyML perfomant real time mashine learning libraryes in Go

FlyML perfomant real time mashine learning libraryes in Go simple & perfomant logistic regression (~100 LoC) Status: WIP! Validated on mushrooms datas

Vadim Kulibaba 1 May 30, 2022
Go (Golang) encrypted deep learning library; Fully homomorphic encryption over neural network graphs

DC DarkLantern A lantern is a portable case that protects light, A dark lantern is one who's light can be hidden at will. DC DarkLantern is a golang i

Raven 2 Oct 31, 2022
A tool for building identical machine images for multiple platforms from a single source configuration

Packer Packer is a tool for building identical machine images for multiple platforms from a single source configuration. Packer is lightweight, runs o

null 2 Oct 3, 2021
Deploy, manage, and scale machine learning models in production

Deploy, manage, and scale machine learning models in production. Cortex is a cloud native model serving platform for machine learning engineering teams.

Cortex Labs 7.9k Dec 30, 2022
This is the repository for the LinkedIn Learning course Learning Go.

Learning Go This is the repository for the LinkedIn Learning course Learning Go. The full course is available from LinkedIn Learning. What is Go? Go i

Zhenguan Tang 0 Nov 2, 2021
This repository is where I'm learning to write a CLI using Go, while learning Go, and experimenting with Docker containers and APIs.

CLI Project This repository contains a CLI project that I've been working on for a while. It's a simple project that I've been utilizing to learn Go,

Tamir Arnesty 0 Dec 12, 2021