Standard machine learning models

Related tags

Machine Learning cog
Overview

Cog: Standard machine learning models

Define your models in a standard format, store them in a central place, run them anywhere.

  • Standard interface for a model. Define all your models with Cog, in a standard format. It's not just the graph – it also includes code, pre-/post-processing, data types, Python dependencies, system dependencies – everything.
  • Store models in a central place. No more hunting for the right model file on S3. Cog models are in one place with a content-addressable ID.
  • Run models anywhere: Cog models run anywhere Docker runs: your laptop, Kubernetes, cloud platforms, batch processing pipelines, etc. And, you can use adapters to convert the models to on-device formats.

Cog does a few things to make your life easier:

  • Automatic Docker image. Define your environment with a simple format, and Cog will generate CPU and GPU Docker images using best practices and efficient base images.
  • Automatic HTTP service. Cog will generate an HTTP service from the definition of your model, so you don't need to write a Flask server in the right way.
  • No more CUDA hell. Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combos are compatible and will pick the right versions for you.

How does it work?

  1. Define how inferences are run on your model:
import cog
import torch

class ColorizationModel(cog.Model):
    def setup(self):
        self.model = torch.load("./weights.pth")

    @cog.input("input", type=Path, help="Grayscale input image")
    def run(self, input):
        # ... pre-processing ...
        output = self.model(processed_input)
        # ... post-processing ...
        return processed_output
  1. Define the environment it runs in with cog.yaml:
model: "model.py:ColorizationModel"
environment:
  python_version: "3.8"
  python_requirements: "requirements.txt"
  system_packages:
   - libgl1-mesa-glx
   - libglib2.0-0
  1. Push it to a repository and build it:
$ cog build
--> Uploading '.' to repository http://10.1.2.3/colorization... done
--> Building CPU Docker image... done
--> Building GPU Docker image... done
--> Built model b6a2f8a2d2ff

This has:

  • Created a ZIP file containing your code + weights + environment definition, and assigned it a content-addressable SHA256 ID.
  • Pushed this ZIP file up to a central repository so it never gets lost and can be run by anyone.
  • Built two Docker images (one for CPU and one for GPU) that contains the model in a reproducible environment, with the correct versions of Python, your dependencies, CUDA, etc.

Now, anyone who has access to this repository can run inferences on this model:

$ cog infer b6a2f8a2d2ff -i @input.png -o @output.png
--> Pulling GPU Docker image for b6a2f8a2d2ff... done
--> Running inference... done
--> Written output to output.png

It is also just a Docker image, so you can run it as an HTTP service wherever Docker runs:

$ cog show b6a2f8a2d2ff 
...
Docker image (GPU):  registry.hooli.net/colorization:b6a2f8a2d2ff-gpu
Docker image (CPU):  registry.hooli.net/colorization:b6a2f8a2d2ff-cpu

$ docker run -d -p 8000:8000 --gpus all registry.hooli.net/colorization:b6a2f8a2d2ff-gpu

$ curl http://localhost:8000/infer -F [email protected]

Why are we building this?

It's really hard for researchers to ship machine learning models to production. Dockerfiles, pre-/post-processing, API servers, CUDA versions. More often than not the researcher has to sit down with an engineer to get the damn thing deployed.

By defining a standard model, all that complexity is wrapped up behind a standard interface. Other systems in your machine learning stack just need to support Cog models and they'll be able to run anything a researcher dreams up.

At Spotify, we built a system like this for deploying audio deep learning models. We realized this was a repeating pattern: Uber, Coinbase, and others have built similar systems. So, we're making an open source version.

The hard part is defining a model interface that works for everyone. We're releasing this early so we can get feedback on the design and find collaborators. Hit us up if you're interested in using it or want to collaborate with us. We're on Discord or email us at [email protected].

Install

No binaries yet! You'll need Go 1.16, then run:

make install

This installs the cog binary to $GOPATH/bin/cog.

Next steps

Issues
  • No CUDA runtime is found / Found no NVIDIA driver on your system.

    No CUDA runtime is found / Found no NVIDIA driver on your system.

    When running a GPU cog:

    $ sudo cog predict r8.im/allenhung1025/[email protected]:f5cd715e99046e0513fe2b4034e8f7d8c102525b02f49efb52b05f46fcb9ea83
    Starting Docker image r8.im/allenhung1025/[email protected]:f5cd715e99046e0513fe2b4034e8f7d8c102525b02f49efb52b05f46fcb9ea83 and running setup()...
    No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
    Traceback (most recent call last):
    ...
    AssertionError: 
    Found no NVIDIA driver on your system. Please check that you
    have an NVIDIA GPU and installed a driver from
    http://www.nvidia.com/Download/index.aspx
    ⅹ Failed to get container status: exit status 1
    

    But CUDA is fine:

    $ cd /usr/local/cuda/samples/1_Utilities/deviceQuery
    $ sudo make
    $ ./deviceQuery
    Detected 1 CUDA Capable device(s)
    
    Device 0: "NVIDIA GeForce RTX 2080 Ti"
      CUDA Driver Version / Runtime Version          11.4 / 11.2
      CUDA Capability Major/Minor version number:    7.5
      Total amount of global memory:                 11016 MBytes (11551440896 bytes)
      (68) Multiprocessors, ( 64) CUDA Cores/MP:     4352 CUDA Cores
    ...
    Result = PASS
    
    opened by bwhitman 12
  • Include timestamps in Redis responses

    Include timestamps in Redis responses

    Timestamps are useful for calculating how long a prediction has taken to run. There are other ways we could get this data (using HTTP streams instead of queueing, and doing accurate timing from the outside), but this is probably the simplest to get started with and arguably the simplest for all sorts of users in the future.

    Signed-off-by: Dominic Baggott [email protected]

    Resolves #589

    opened by evilstreak 8
  • build error

    build error

    when I build a model with cog, I always get this error and fail to build. How can I handle this?

     => ERROR [stage-0  3/12] RUN curl https://pyenv.run | bash &&  git clone https://github.com/momo-lab/pyenv-install-latest.git "$(pyenv root)"/plugin  125.3s
    ------
     > [stage-0  3/12] RUN curl https://pyenv.run | bash &&         git clone https://github.com/momo-lab/pyenv-install-latest.git "$(pyenv root)"/plugins/pyenv-install-latest &&        pyenv install-latest "3.8.2" &&         pyenv global $(pyenv install-latest --print "3.8.2"):
    #8 1.093   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
    #8 1.093                                  Dload  Upload   Total   Spent    Left  Speed
    100   270  100   270    0     0    102      0  0:00:02  0:00:02 --:--:--   102
    #8 3.785 curl: (7) Failed to connect to raw.githubusercontent.com port 443: Connection refused
    #8 3.788 /bin/sh: 1: pyenv: not found
    #8 3.811 Cloning into '/plugins/pyenv-install-latest'...
    #8 124.1 fatal: unable to access 'https://github.com/momo-lab/pyenv-install-latest.git/': gnutls_handshake() failed: The TLS connection was non-properly terminated.
    ------
    executor failed running [/bin/sh -c curl https://pyenv.run | bash &&    git clone https://github.com/momo-lab/pyenv-install-latest.git "$(pyenv root)"/plugins/pyenv-install-latest &&        pyenv install-latest "3.8.2" &&         pyenv global $(pyenv install-latest --print "3.8.2")]: exit code: 128
    ⅹ Failed to build Docker image: exit status 1
    
    
    opened by Hurricane-eye 8
  • Initial working version using Pydantic for type annotations

    Initial working version using Pydantic for type annotations

    The exploration in #343 was sufficiently explored and getting very messy, so I thought I would create a new pull request for the real implementation.

    This is an implementation of #193, #205, and #259 using Pydantic for the type system, FastAPI for the model server, and OpenAPI for the schema.

    Models now look a bit like this:

    from cog import Predictor, Input, Path
    import torch
    
    class ColorizationPredictor(Predictor):
        def setup(self):
            self.model = torch.load("./weights.pth")
    
        def predict(self,
              image: Path = Input(title="Grayscale input image")
        ) -> Path:
            processed_input = preprocess(image)
            output = self.model(processed_input)
            return postprocess(output)
    

    The HTTP model server is a work in progress and will be finalized as part of future work.

    The Redis queue worker remains in place and the API is unchanged. As part of future work we will implement an AMQP API with a better API.

    Commits are significant and this should be reviewed commit by commit. This is an intentionally large and long-running branch -- main is currently used as the documentation for Replicate.

    Todo

    • [x] cog predict with new API
    • [x] Input ordering?
    • [x] Ensure HTTP server is single threaded.
    • [x] HTTP URLs for input and output
    • [x] Add choices option to Input to generate an Enum.
    • [x] cog init template
    • [x] Update getting started docs
    • [x] Update reference docs
    • [x] Rename Predictor to BasePredictor so we can do from cog import BasePredictor, Input, Path without colliding with a Predictor subclass.
    • [x] Better errors if you get output type wrong.
    • [x] ~Reliably set Cog version. In development, the version is set to 0.0.1 which is pretty useless. It should probably be hardcoded somewhere, and the release process needs updating. We need this to be able to switch between old and new Cog API in Replicate.~ We're just going to support version dev everywhere which means "latest".
    • [x] Update examples https://github.com/replicate/cog-examples/pull/8
    • [x] Update the OpenAPI Outputs to be a consistent place. (Simple outputs currently end up in Request object schema)
    • [x] Test returning object as output
    • [x] Test arbitrary JSON objects in output
    • [x] Is go.sum up to date?

    Future (Pydantic + OpenAPI)

    • [ ] Tell users to upgrade if they use @cog.input() with new Cog.
    • [ ] Document nouns to make sure we're all on the same page. "Input"/"Inputs"?
    • [ ] Document how to migrate @input() to Pydantic type annotations.
    • [ ] Go through and fix any TODOs
    • [ ] Make input validation errors work for cog predict in a simple way until we lock down HTTP API
    • [ ] Make input errors work for Redis API & Replicate
    • [ ] Integrate with Replicate - https://github.com/replicate/replicate-web/pull/907
    • [ ] Force users to pick an output type -- throw an error if it's not set. (Couldn't just be Any, but they need to be explicit!)
    • [ ] Clean up temporary files -- both input and output
    • [ ] Do we want to make cog predict with an existing image backwards compatible? ('cos there are old models on Replicate)
    • [ ] What happens if you use pathlib.Path as an input type? Does it work? Should we throw an error? (This might happen if you import wrong package / collide imports)
    • [ ] What happens if you return pathlib.Path from a predict function? Does this work? Should it throw an error?
    • [ ] Should we have a fixed set of supported inputs/outputs somehow?
    • [ ] Do we want to force users to put extra properties in additionalProperties? Should we even allow extra properties to cog.Input()?
    • [ ] Merge examples PR https://github.com/replicate/cog-examples/pull/8

    Far-flung future

    • Vendor Pydantic and FastAPI
    • Add back support for PyTorch and Tensorflow tensors as output. This was removed because it was fiddly to get working with Pydantic. You now need to convert them to numpy arrays.
    • Add back timing information to server.
    • Better error if you forget an output type
      pydantic.error_wrappers.ValidationError: 1 validation error for Prediction
      output
        unexpected value; permitted: None (type=value_error.const; given=hello foo; permitted=(None,))
      
    • Multiple file outputs for cog predict. It currently just outputs the plain JSON if there isn't a single file output.
    • Logging, as described in comment in #343
    • Review request/response objects and make sure we're happy with them.
      • This might be an opportunity to rename the statuses and ensure they're consistent ("success" vs "failed").
      • Do we want the input type to have a second input level so we can have top-level request options?
    • Do we need separate URLs for input/output schema, or can they be reliably fetched from OpenAPI?
    • [ ] Finalize & document HTTP API
    • [ ] Do we want to version the prediction API, or let the client handle it? https://github.com/replicate/cog/issues/94
    • [ ] Test arbitrary objects in input
    • [ ] What if people run new models from a directory of code with old Cog? What happens if we do that? Do we care? #286
    • [ ] What if people run new models from an image with old Cog? Maybe we throw a simple error from /predict?

    Based on (& blocked by)

    • #383
    • #385

    Refs

    Closes #58 Closes #113 Closes #193 Closes #205 Closes #212 Closes #246 Closes #259 Closes #262 Closes #263 Closes #304 Closes #306 Closes #327 Closes #328

    opened by bfirsh 8
  • Throw error if invalid keys exist in cog.yaml

    Throw error if invalid keys exist in cog.yaml

    Fix for:

    Throw error if invalid keys exist in cog.yaml #34

    Approaches:

    1) Use hard-coded key pair to test the yaml file

    Create a list of key pairs or a struct to validate the cog.yaml file

    Pros:

    • Simple & straight forward approach
    • Easy to test and validate it works

    Cons:

    • Hard to implement complex logic
    • Needs more code changes
    • Need to learn go and have knowledge of internal systems to make changes
    • Hard to keep track & validate the different versions of cog.yaml file

    2) Use a JSON Schema to validate

    This approach uses a JSON Schema to validate the cog.yaml file.

    Pros:

    • Uses Industry standard approach to validate the yaml file
    • Most cases will need a simple change to json schmema
    • No need to learn go to understand various fields in cog.yaml
    • The json-schema serves as the documentation.

    Cons:

    • Dependencies on external libraries and validation mechanism
    • Would be hard to understand for people not familiar with jsonSchemas
    • Required editing a large json file to make changes

    Approach Selected: "Using JSONSchema"

    Decided to implement the jsonschema approach based on the pros & cons and especially keeping the end-user in mind and the "open-source" nature of the project.

    The approach will not enforce new rules but only validates that the keys present in the yaml are defined in the schema.

    How to generate the initial JSON schema

    • Create a yaml file containing all the fields present in the cog.yaml
    • Converted the yaml to JSON using this tool: https://onlineyamltools.com/convert-yaml-to-json
    • Then generate the jsonschema using this tool: https://www.jsonschema.net/home

    The schema required few changes:

    • Remove the required field
    • Allow both number and string for python_version field

    Validate command

    The PR also includes a validate command to verify any cog.yaml in the current directory.

    Usage:

    cog validate
    

    Examples:

    Empty cog.yaml

    No error will be thrown and cog will use the default schema generated by the below code:

    func DefaultConfig() *Config {
    	return &Config{
    		Build: &Build{
    			GPU:           false,
    			PythonVersion: "3.8",
    		},
    	}
    }
    
    

    These are all valid cog.yaml files:

    • No File
    • Empty file
    • build:

    Bad yaml file example & errors

    cog.yaml:

    buildd:
    

    Output: ⅹ (root) Additional property builds is not allowed

    cog.yaml:

    buildd:
      python_version: "3.8"
      python_packages:
        - "torch=1.8.0"
    

    Output: ⅹ (root) Additional property buildd is not allowed

    cog.yaml:

    build:
      python_versions: "3.8"
      python_packages:
        - "torch=1.8.0"
    

    Output:

    ⅹ build Additional property python_versions is not allowed ..........................................................................................

    Signed-off-by: Shashank Agarwal 2 [email protected]

    opened by imshashank 8
  • Design & document HTTP API

    Design & document HTTP API

    This is a work-in-progress implementation of #413. I was considering creating an issue to discuss the design of the HTTP API, but I figure a pull request and some documentation might be the most productive medium for doing this.

    The text of the documentation is incomplete and might be partial sentences. Mainly looking here for feedback on the API, then we can complete all the sentences.

    I think it is mostly there, but a few design questions:

    1. Do we like our request and response objects?
    2. Do we like our status names? (Maybe failure instead of failed? Is there some prior art here we can build upon instead of inventing something new?)
    3. #428
    4. Is this compatible with #437 and #443?
    5. File handling is a work in progress and I will create a separate piece of documentation about that.
    6. Is this compatible with a potential future async version?
    7. Do we want to version it? #94
    opened by bfirsh 7
  • Show a replicate.com link after pushing to r8.im

    Show a replicate.com link after pushing to r8.im

    Show a message after a successful push. If the push was to the Replicate registry then also show a link to the model page on replicate.com.

    Screenshot 2022-02-28 at 17 51 48

    (It's using localhost:8100 as the replicate registry in development – in the wild that'd be matching against and replacing r8.im)

    The path not taken

    I talked to @bfirsh about displaying a message from replicate.com rather than hardcoding this into Cog. Unfortunately, there's no existing call made to replicate.com after the docker push command has completed, and there's no way I know of to make docker push display more info by feeding it a suitable response from the server.

    So, although it's much harder to update this when it's in Cog (because rolling out new code changes requires people to update their version), this seems like the right place to have it for now.

    Fixes #210

    opened by evilstreak 7
  • Proposed design for public AMQP queueing API

    Proposed design for public AMQP queueing API

    Context

    We currently have a private queue-based redis API that replicate.com uses to talk to Cog models, and we want to turn that into a well-defined public API that uses standard protocols (in this case AMQP).

    Requirements

    • Run predictions from JSON inputs in a request queue
    • Put JSON output of prediction on a response queue specified in request
    • Read input files from a URL (HTTP/S3/GCS) and write output files to particular location in an S3/GCS bucket
    • Record log output of prediction, in some way that can be read as the prediction is running
    • Put in-progress outputs from the model onto the response queue, as the model is running
    • Validate inputs and outputs based on a JSON Schema
    • Include metrics about run in response: model start up time, request processing time, and system metrics like CPU, RAM, and GPU memory used.
    • Errors and logs in response if prediction fails.
    • Cancel a running prediction.

    Strawman design

    At a high-level, the AMQP API uses the same request and response objects as the HTTP API, defined with an OpenAPI schema. But, instead of HTTP requests and responses, these objects are sent over AMQP request and response queues.

    Requests

    • The message body is the same format as the HTTP API request body being implemented in #378. It contains the input and any other metadata.
    • Message body encoded as JSON (content_type property set to application/json).
    • The response queue is specified with the reply_to property on the message.
    • Files are represented as URIs, prefixed with either file://, http://, https://, s3://, or gs://.
    • Example request body:
    {
      "input": {
        "image": "s3://hooli-input-bucket/hotdog.jpg",
        "disentanglement_threshold": 5
      },
      "output_file_prefix": "s3://hooli-output-bucket/5245/"
    }
    

    Responses

    • The message body is the same format as the HTTP API response body. It contains the status, output, logs, metrics, and other metadata.
    • Files are represented as URIs, like in the response. The model will upload them to output_file_prefix defined in the request. Credentials are provided at runtime via environment variables.
    • The response will be sent to the reply_to queue specified in the request.
    • Message body encoded as JSON (content_type property set to application/json).
    • Includes any message IDs present in the request (message_id, correlation_id properties).
    • Includes timing information and other performance metrics (setup time, prediction time, CPU time, memory usage, etc).
    • Includes logs. For each log message, or some log message updating frequency, this message will be sent multiple times, with status: "processing".
    • For models that return in-progress output as as they are running, this message will be sent multiple times, with status: "processing" and the "output" key set to the output.
    • Example request body:
    {
      "status": "success",
      "output": "s3://hooli-output-bucket/5245/output.jpg",
      "metrics": {
        "setup_time": <time>,
        "prediction_time": <time>
      },
      "logs": <array of strings, one per line>
    }
    

    Schema

    The AMQP API is specified with a subset of the OpenAPI/JSON Schema used for the HTTP API.

    It is available both as a org.cogmodel.openapi_schema label on the image, and at /openapi.json in the HTTP API.

    The relevant subset looks like this:

    {
      "components": {
        "schemas": {
          "Input": {
            "title": "Input",
            "required": ["image", "disentanglement_threshold"],
            "type": "object",
            "properties": {
              "image": {
                "description": "Image to manipulate",
                "type": "string",
                "format": "uri",
                "x-order": 0
              },
              "disentanglement_threshold": {
                "type": "number",
                "x-order": 1
              }
            }
          },
          "Output": {
            "title": "Output",
            "type": "string",
            "format": "uri"
          }
        }
      }
    }
    
    

    Big differences between old and new API

    • New system would use a well-defined, public AMQP API instead of the current system of using redis queues, to run predictions. Rationale: want to use standard tools, and make it and easy to run predictions in other workflows.
    • Old system had separate queues for responses, logs, and timing info, new system puts all of that info in the response queue. Rationale: easier to keep track of 1 queue?

    Todo

    • Cancel running predictions
    • Error handling
    • Pass credentials for file downloads/uploads with a request for scoping
    • Backwards compatibility on Replicate

    Some references worth reading:

    Edited by @preeth1, @bfirsh, ... (maintainers -- add your name here! consider this is a wiki.)

    opened by preeth1 7
  • refactor: re-usable config loading, and basic error definitions

    refactor: re-usable config loading, and basic error definitions

    What does this PR do?

    • Standardises config-related functionality in utils/config.go
    • Eliminates some repetitious config loading code
    • Adds a func to walk the parent directories in search of a cog.yaml
    • Adds some basic error creation code in pkg/errors/errors.go

    Next Steps/Questions

    • Is this a reasonable approach?
    • ~~How do we want cog run to work - relative to cwd or relative to the root (see #102)?~~
    • ~~What else needs to change to allow cog usage in subdirectories?~~
    • What's the best way to test this? Unit tests (where?), some addition to the e2e tests?

    (If you like the config-loading (utils/config.go) refactor, we could merge that as a separate PR first to keep PRs small and lean)

    opened by synek 7
  • cog init

    cog init

    This PR introduces a new CLI command cog init that can be used to bootstrap an existing project with starter cog.yaml and predict.py files.

    $ cog init
    Setting up the current directory for use with Cog...
    Created /Users/z/git/replicate/cog/cog.yaml
    Created /Users/z/git/replicate/cog/predict.py
    

    To Do

    • [x] Make sure I'm Doing It Right™, Golang-wise
    • [x] Do not overwrite files if they already exist
    • [x] Refine the cog.yaml and predict.py templates to contain the right starter content
    • [x] Write docs
    • [x] Write tests

    Stretch goals

    • [ ] Derive python_packages somehow (pipreqs? look for requirements.txt?)
    • [ ] Add example script/download-weights?
    opened by zeke 6
  • docker: Error response from daemon: invalid mount config for type

    docker: Error response from daemon: invalid mount config for type "bind": stat /my/current/folder: permission denied.

    Hi, Thanks for the cool work!

    I'm following the tutorial here and working on Ubuntu as a non-root user. But when I run cog run python, I encountered an error:

    Running 'python' in Docker with the current directory mounted as a volume...
    docker: Error response from daemon: invalid mount config for type "bind": stat  /my/current/folder: permission denied.
    

    It seems related to the mount setting of docker, but I'm not familiar with the docker setting. How can I deal with the error? Where can I find the Dockerfile that cog created and modify the setting? Thanks!


    The full building log:

    Building Docker image from environment in cog.yaml...
    [+] Building 3.8s (12/12) FINISHED                                                                                                                                                                                             
     => [internal] load build definition from Dockerfile                                                                                                                                                                      0.0s
     => => transferring dockerfile: 489B                                                                                                                                                                                      0.0s
     => [internal] load .dockerignore                                                                                                                                                                                         0.0s
     => => transferring context: 2B                                                                                                                                                                                           0.0s
     => resolve image config for docker.io/docker/dockerfile:1.2                                                                                                                                                              1.8s
     => CACHED docker-image://docker.io/docker/dockerfile:[email protected]:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2230a333fdbcc                                                                                         0.0s
     => [internal] load metadata for docker.io/library/python:3.8                                                                                                                                                             1.3s
     => [internal] load build context                                                                                                                                                                                         0.0s
     => => transferring context: 13.61kB                                                                                                                                                                                      0.0s
     => [stage-0 1/4] FROM docker.io/library/python:[email protected]:c2842aababbe377f9c76f172d9eb39487e23f306b2f29f020f3f6654cb0876e9                                                                                               0.0s
     => CACHED [stage-0 2/4] COPY .cog/tmp/cog-0.0.1.dev-py3-none-any.whl /tmp/cog-0.0.1.dev-py3-none-any.whl                                                                                                                 0.0s
     => CACHED [stage-0 3/4] RUN --mount=type=cache,target=/root/.cache/pip pip install /tmp/cog-0.0.1.dev-py3-none-any.whl                                                                                                   0.0s
     => CACHED [stage-0 4/4] WORKDIR /src                                                                                                                                                                                     0.0s
     => exporting to image                                                                                                                                                                                                    0.0s
     => => exporting layers                                                                                                                                                                                                   0.0s
     => => writing image sha256:ea7f49cf567ab25ac6b8d6c8a6abb3d6bb6c1738b4a2c08ff19dd75536e42811                                                                                                                              0.0s
     => => naming to docker.io/library/cog-cogtest-base                                                                                                                                                                       0.0s
     => exporting cache                                                                                                                                                                                                       0.0s
     => => preparing build cache for export                                                                                                                                                                                   0.0s
    
    Running 'python' in Docker with the current directory mounted as a volume...
    docker: Error response from daemon: invalid mount config for type "bind": stat /home/annahung/189/my_project/playground/cog_test: permission denied.
    See 'docker run --help'.
    ⅹ exit status 126
    
    
    opened by annahung31 6
  • One liner installation not working for aarch64

    One liner installation not working for aarch64

    Installing via

    sudo curl -o /usr/local/bin/cog -L https://github.com/replicate/cog/releases/latest/download/cog_`uname -s`_`uname -m`
    

    did not work for me, as uname -m is aarch64 and not arm64 and there is no release for aarch64 which is only an alias.

    OS is an ARM Ubuntu:

    Distributor ID:	Ubuntu
    Description:	Ubuntu 20.04.4 LTS
    Release:	20.04
    Codename:	focal
    
    opened by foxriver76 0
  • Bump numpy from 1.21.5 to 1.22.0

    Bump numpy from 1.21.5 to 1.22.0

    Bumps numpy from 1.21.5 to 1.22.0.

    Release notes

    Sourced from numpy's releases.

    v1.22.0

    NumPy 1.22.0 Release Notes

    NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

    • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
    • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
    • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
    • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
    • A new configurable allocator for use by downstream projects.

    These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

    The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

    Expired deprecations

    Deprecated numeric style dtype strings have been removed

    Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

    (gh-19539)

    Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

    numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

    (gh-19615)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the Security Alerts page.
    dependencies python 
    opened by dependabot[bot] 0
  • Terminate worker if it can't receive messages

    Terminate worker if it can't receive messages

    When the worker can't receive messages because, for example, it cannot connect to Redis, the current behaviour is to output a stack trace to stderr and try again. It does this continuously, in a tight loop, which generates an awful lot of log output and disguises the fact that it's not working properly.

    A more useful behaviour would be for the worker to crash when it can't connect, so the container shows as non-running. That's what this change does.

    A few questions for reviewers:

    • Is this feature even desirable? We need to not uncontrollably dump stack traces to stderr, and it'd be nice to have some sort of strong signal that something is wrong, but there are probably other ways to handle both of those needs. Is this a reasonable way to approach it?
    • Should we have some sort of sleep/backoff in that failure loop? This will currently exit very quickly – do we want to be able to stay running through Redis blips of up to 15s? 60s? 300s?
    • Should we try connecting to Redis before we run setup(), so we can crash early before spending potentially a few minutes working?
    • What would a reasonable way to test this be? Try running it via Docker, wait a few seconds, and check the container has exited?
    opened by evilstreak 0
  • Tight loop causes redis worker to consume entire CPU while waiting for messages

    Tight loop causes redis worker to consume entire CPU while waiting for messages

    The loop here takes a full CPU while async polling for messages from Redis: https://github.com/replicate/cog/blob/main/python/cog/server/redis_queue.py#L223-L224

    Can we do this with a blocking pop instead?

    type/bug 
    opened by andreasjansson 0
  • Cog + Codespaces

    Cog + Codespaces

    GitHub Codespaces makes it easy to start development environments from GitHub repositories. Lots of cog.yamls are popping up in GitHub repositories, defining the development environment for ML models. These two things should be combined somehow to make one-click development environments for ML models!

    How it works right now

    If you boot up a Codespace with cog.yaml it boots up into the default Codespaces environment. If you install Cog, Docker is available and you can run Cog commands as you might expect.

    This mostly works except:

    • Cog is not installed in the Codespace by default.
    • Python dependencies aren't available for code completion, debugging, etc.
    • You can't run a Python script in the terminal because none of the dependencies will be installed.
    • You can't run a Jupyter notebook in VSCode because none of the dependencies will be installed.

    Architecture

    At a high level, we have a few options:

    1. Run Cog inside the Codespaces environment

    This is along the lines of what works at the moment. We then make a best-effort attempt to install all of the dependencies inside the default Codespaces environment.

    Advantages:

    • Simplicity.
    • cog commands just work as you expect.

    Disadvantages:

    • The environment in Codespaces won't be quite right. The dependencies might not be installed exactly the same as they are inside Cog, so things will break for weird, hard-to-debug reasons. The CUDA versions might not be set up correctly so things will break in horrible CUDAry ways. If you just run python train.py or open a Jupyter notebook it'll work 80% of the time, but won't work 20% of the time.

    2. Run Codespaces inside the Cog environment

    We use cog.yaml to build the Docker image the Codespaces environment uses.

    Advantages:

    • Beautiful perfect reproducible environment. Everything in the Codespaces environment is just so.

    Disadvantages:

    • The right hooks to build the Docker images from cog.yaml aren't in Codespaces, which makes this complicated.
    • You can't run cog predict, cog push, etc in the terminal.

    3. Both??

    Maybe we could build the Codespaces environment using cog.yaml, and make Cog available inside that environment? Would this work?

    Strawman first step

    This is all very complicated. Maybe as a simple first step we just get the Python dependencies working in the current setup we have.

    We're intending to support requirements.txt in cog.yaml and probably make it the recommended way to define Python dependencies. The default Codespaces environment installs Python dependencies from requirements.txt.

    If both systems use requirements.txt, this should all just work!

    Strawman user journey

    Given this implementation, this could work...

    I am an ML researcher and I've come across an ML model on GitHub that I want to fine-tune and tinker with. Here's what I do:

    • There is a cog.yaml in a GitHub repository
    • I click Code -> Create Codespace.
    • The readme tells me to download some pretrained weights: script/download-weights
    • The readme says I can run predictions by running cog predict -i @input.jpg. It does what I expect.
    • I run cog run python finetune.py mydata/ to fine-tune the model on my own data.
    • When I go to edit the code in the editor, it has all the completions and useful things I expect from VSCode.
    • I run cog push r8.im/bfirsh/superbnet to push up my fine-tuned model to Replicate.

    The notable things that don't work so well:

    • If I open a notebook, it mostly works, but some stuff might be janky because the dependencies are different.
    • If I run python finetune.py without the cog run prefix by accident it mostly works, but some stuff might not work right.

    Next steps

    • [ ] We implement requirements.txt support and make it the recommended method of defining Python dependencies. #157
    • [ ] Figure out some way of getting Cog installed in the Codespace. With another configuration? Get it in the Codespaces base image?
    type/epic 
    opened by bfirsh 0
  • Local web interface

    Local web interface

    It'd be neat if Cog served a Replicate-style web interface when you ran the web server for testing locally.

    User data

    • As requested by @postera-james
    opened by bfirsh 4
Releases(v0.3.10)
  • v0.3.10(Jun 22, 2022)

    Changes

    A bumper release, with five improvements to the queue worker:

    • We've added some OpenTelemetry instrumentation for distributed tracing #646
    • Start and finish timestamps are includes with responses, in an experimental property #599
    • The worker handles a SIGTERM better, shutting down more quickly #658
    • A response is sent when the worker starts a prediction, for quicker feedback #657
    • A fix to support output like Iterator[List[Output]] #653

    Full changelog

    • 3787884 Add HTTP test for iterator of list of complex output
    • a6f002c Add OpenTelemetry instrumentation to queue worker
    • ce434ce Add docs for queue worker telemetry
    • f0d45d8 Bump github.com/spf13/cobra from 1.4.0 to 1.5.0
    • df737d5 Bump github.com/stretchr/testify from 1.7.2 to 1.7.4
    • 9848edf Fix bad variable in log line
    • 3b1553f Handle SIGTERM gracefully
    • 61c5fc4 Ignore OpenTelemetry typing issues
    • da5f485 Include timestamps in Redis responses
    • b3bec59 Pin protobuf to make opentelemetry work
    • cbfe5bf Record time in queue in opentelemetry span
    • ba12047 Remove unused variable: start_time
    • c104704 Send response when queue worker starts predict
    • fcbe521 Split encode_json() into make_encodeable() and upload_files()
    • fa3a472 Use blpop not brpop in Redis tests
    • 9a20b26 Use make_encodeable inside prediction runner
    • 01bd1bb document how to optionally set output properties
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.32 MB)
    cog_Darwin_x86_64(9.52 MB)
    cog_Linux_arm64(8.43 MB)
    cog_Linux_x86_64(8.82 MB)
  • v0.3.9(Jun 14, 2022)

  • v0.3.8(Jun 8, 2022)

  • v0.3.6(Jun 7, 2022)

  • v0.3.5(Jun 1, 2022)

  • v0.3.4(May 19, 2022)

    Changes

    • When passing the -i option to cog predict without the name of an input, it now defaults to the first. Previously, it would default to magic input called input.

    Full changelog

    • deb4c04 Add mypy to vscode and lint
    • 8fd6c77 Appease mypy gods
    • f1ed980 Bump github.com/docker/cli
    • 8b3a613 Bump github.com/docker/cli
    • 585f8eb Bump github.com/docker/docker
    • 9c0b86a Bump github.com/docker/docker
    • 2712640 Bump github.com/golangci/golangci-lint from 1.45.2 to 1.46.2
    • ab35949 Bump gotest.tools/gotestsum from 1.8.0 to 1.8.1
    • 57354dd Fix type for choices
    • 04726d5 Fixes #592
    • 82a569c Make default input first input
    • 96d1a8f Require typing-extensions>=4.1.0
    • 25cfd50 Update README.md
    • 1323692 Update pkg/cli/init-templates/predict.py
    • 765b4a1 Update remote image test to recent Cog model
    • 498fe29 Update test-integration/test_integration/test_predict.py
    • 747b513 Updates formatting in getting-started, drops --gpus
    • 92a70b1 Upgrade to Go 1.18
    • 2a725db go mod tidy
    • 159cfc7 only catch AttributeError
    • 7e96884 return str(t) if there's no qualname in t
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(8.97 MB)
    cog_Darwin_x86_64(9.15 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.47 MB)
  • v0.3.3(May 3, 2022)

  • v0.3.2(Apr 30, 2022)

  • v0.3.1(Apr 27, 2022)

  • v0.3.0(Apr 20, 2022)

    Hightlights

    Cog now has an experimental ⚡️ built-in queue worker ⚡️ that can process predictions from a Redis queue. See docs/redis.md.

    This release also includes lots of bugfixes and improvements to the documentation.

    Breaking changes

    • 311ffc0 The success status has been change to succeeded in the HTTP API and throughout Cog

    Changelog

    • 1cde0ae Add experimental notice to queue API
    • 3e1447b Add logs to queueing API responses
    • e010d1e Allow version to be specified with make install
    • 5d82ba0 Document that file handling is subject to change
    • 0f7128f Document the Redis queue API
    • 1ef7bbc Don't expect prediction ID from queueing API request
    • 5e265ab Fix curl command and add real image
    • 0defd8b Fix enums being passed to predict() when using choices
    • bacaf22 Fix getting started guide
    • 290d987 Include output and logs with failed responses
    • 90c78e7 Remove input wrapping for queueing API
    • af91725 Remove redis_log_capture
    • 9b69a58 Rename inputs to input for queueing API requests
    • 4f2271c Rename output key from "value" to "output"
    • 311ffc0 Rename success status to succeeded
    • 55875b0 Rework predictor subprocess to be long running
    • 4d3bff2 Support complex outputs from predictors
    • 5e1824e Use an Enum for whether output is a generator
    • 5c7201b fix notebook docs
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.25 MB)
    cog_Darwin_x86_64(9.45 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.78 MB)
  • v0.2.1(Apr 15, 2022)

  • v0.2.0(Apr 4, 2022)

    Changelog

    • 32865a1 Add EXPOSE to Dockerfile (#517)
    • f1ca5cc Add Python 3.10 support for PyTorch
    • 4591e1c Add documentation to explain how to deploy models (#526)
    • e05b0c1 Bump github.com/docker/docker
    • bb64680 Bump github.com/getkin/kin-openapi from 0.92.0 to 0.93.0 (#512)
    • da71b5d Bump github.com/getkin/kin-openapi from 0.93.0 to 0.94.0
    • 9ad0a58 Define some nouns
    • e7abca8 Document HTTP API
    • 3e1e608 Fetch latest torch versions from PyPi
    • d1426c0 Fix output documentation
    • 45f351a Install pyenv from github (#525)
    • 6724526 Make it clear what is incomplete (#528)
    • 26c82a0 More forgiving parser to deal with --extra-index-url
    • d73af5c Pick latest patch version for CUDA
    • 0528f3c Relax generate compatibility to handle ROCm
    • dadab64 Rename run.cog.cog_version label to run.cog.version (#532)
    • e855d3d Revert "Update compatibility matrices (#504)" (#530)
    • cb707f7 Switch to cog.run labels
    • b057e19 Update compatibility matrices
    • 488203e Update compatibility matrices (#504)
    • dd4d325 add -p flag tocog run to publish ports
    • 9c8d6a2 document how to use notebooks with cog (#524)
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.25 MB)
    cog_Darwin_x86_64(9.45 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.78 MB)
  • v0.1.4(Mar 25, 2022)

    Changelog

    • dd9e228 Add Schema test for int enums
    • 50c0295 Bump github.com/docker/cli (#510)
    • 20b3a0e Bump github.com/golangci/golangci-lint from 1.44.2 to 1.45.0 (#501)
    • cad7cfb Bump github.com/golangci/golangci-lint from 1.45.0 to 1.45.2 (#509)
    • 9c7c4a9 Bump github.com/stretchr/testify from 1.7.0 to 1.7.1
    • 556dcfc Bump golang.org/x/tools from 0.1.9 to 0.1.10 (#492)
    • 709c807 Fix int enums
    • 5dc27a2 Update README (#507)
    • e08392e document input and output types (#503)
    • 244a811 fix invalid empty lists from cog init (#493)
    • 7464b27 make heredocs consistent for Request and Response
    • 451882a remove stops from heredocs for consistency
    • 4505b80 s/build, ship, run/package/ (#508)
    • 9c26117 use ge and le options in predictor template
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.23 MB)
    cog_Darwin_x86_64(9.44 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.76 MB)
  • v0.1.3(Mar 15, 2022)

    Changelog

    • 12707a1 Bump github.com/docker/cli
    • eeff3cd Bump github.com/docker/docker
    • afe6826 Bump github.com/getkin/kin-openapi from 0.91.0 to 0.92.0 (#487)
    • e1d5729 Bump github.com/spf13/cobra from 1.3.0 to 1.4.0
    • 34ae4d6 Fix yielding predictor cleanup bug in HTTP server
    • 3f7bfa6 Remove lies (#477)
    • 814f002 Support int choices
    • d39709c document cog debug dockerfile (#474)
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.23 MB)
    cog_Darwin_x86_64(9.44 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.76 MB)
  • v0.1.2(Mar 9, 2022)

    A little release that fixes the choices option to Input and progressive output when there are input files.

    Changelog

    • 7464588 Add redis queue test for inputting and yielding files
    • bc3ea9d Add type to Response.status schema
    • 5184cab Add types to enums in the schema
    • 6c6988d Bump github.com/getkin/kin-openapi from 0.90.0 to 0.91.0
    • 5dec7f2 Fix yielding predictor cleanup bug
    • 4898008 Remove cog.Predictor
    • 5d6197f document how to yield progressive output
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.23 MB)
    cog_Darwin_x86_64(9.44 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.76 MB)
  • v0.1.1(Mar 7, 2022)

    This small release includes support for progressive output. (docs forthcoming).

    👉🏼 If you're upgrading from 0.0.x, see the 0.1.0 release notes for details on how to upgrade your model to be compatible with 0.1.x versions.

    Changelog

    • e5497b8 Fix progressive output being one iteration behind
    • aa30ec7 Make progressive output a list
    • cf94f63 Show a replicate.com link after pushing to r8.im
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.23 MB)
    cog_Darwin_x86_64(9.43 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.76 MB)
  • v0.1.0(Mar 2, 2022)

    👷‍♀️ This new version of Cog includes significant changes from previous versions. If you encounter any problems using it, please open an issue and we'll help you out! :octocat:

    What's changed?

    It's all about the schema! {...}

    Cog's Python API has been updated to use the popular Pydantic library under the hood. Pydantic makes use of Python's type annotations (a native feature of the Python language) to help validate your code and prevent runtime errors. But perhaps most importantly, Pydantic can convert a Python file into a JSON schema.

    Cog now generates a JSON schema from the Predictor you've defined in your predict.py file, and uses another popular Python package called FastAPI to dynamically build an OpenAPI schema and HTTP web server that can be used as a JSON API to generate predictions from your model. And because it's using OpenAPI, this server also dynamically generates documentation specific to your model, and a Swagger UI interface for trying out the API.

    With this new schema-focused design, the inputs and outputs you define for your model are converted into a portable, language-agnostic structured data format. This helps guide us toward a more standardized way of defining model interfaces and paves the way to better documentation, more automated tests, easier integrations across systems, and less reinventing the wheel.

    Upgrading

    First, install Docker if you haven't already. Then run this in a terminal:

    sudo rm $(which cog) # (if you have a previous version of cog installed)
    sudo curl -o /usr/local/bin/cog -L https://github.com/replicate/cog/releases/latest/download/cog_`uname -s`_`uname -m`
    sudo chmod +x /usr/local/bin/cog
    

    Examples

    To get a feel for the changes, check out the replicate/cog-examples repository which has been updated with examples supporting this new version:

    • https://github.com/replicate/cog-examples/blob/main/blur/predict.py
    • https://github.com/replicate/cog-examples/blob/main/hello-world/predict.py
    • https://github.com/replicate/cog-examples/blob/main/resnet/predict.py

    Upgrading from Cog 0.0.x to 0.1.x

    Reference docs for Cog's new Python API can be found here.

    In Cog versions up to 0.0.20, you described inputs using @cog.input decorators on your predict method. For example:

    @cog.input("image", type=Path, help="Image to enlarge")
    @cog.input("scale", type=float, default=1.5, help="Factor to scale image by")
    def predict(self, image, scale):
        ...
    

    From Cog 0.1.0 onwards, we've started using Pydantic to define input and output types. Rather than describing inputs using decorator annotations, you now describe them with Python's built-in type hinting. Here's how you'd define the same inputs now:

    def predict(self,
        image: Path = Input(description="Image to enlarge"),
        scale: float = Input(description="Factor to scale image by", default=1.5)
    ) -> Path:
        ...
    

    The parameters that Input() takes are pretty similar to those @cog.input() used to take. Here are the differences:

    • It no longer takes a type parameter; use a type hint instead.
    • The help parameter has been renamed to description.
    • The options parameter has been renamed to choices.
    • The min option has been replaced with ge (greater than or equal to)
    • The max option has been replaced with le (less than or equal to)

    The other major difference is that you now need to define the output type of your model. That's the -> Path bit in the example above. That might be a simple type like str, float or bool. If you need to handle multiple outputs, check out the new documentation for complex output objects.

    If you have any trouble using this new version of Cog, please open an issue. :octocat:

    Feedback

    This new version of Cog includes a number of internal and user-facing changes from previous versions, so there could be some rough edges.

    If you encounter any problems using it, please open an issue and we'll help you out! :octocat:

    You can also chat with us on Discord at discord.gg/replicate. 💬

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.23 MB)
    cog_Darwin_x86_64(9.43 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.76 MB)
  • v0.1.0-alpha(Feb 17, 2022)

    👷‍♀️ This is a prerelease version of Cog. Please give it a try and file issues if you encounter any problems.

    What's changed?

    It's all about the schema!

    Cog's Python API has been updated to use the popular Pydantic library under the hood. Pydantic makes use of Python's type annotations (a native feature of the Python language) to help validate your code and prevent runtime errors. But perhaps most importantly, Pydantic can convert a Python file into a JSON schema.

    Cog now generates a JSON schema from the Predictor you've defined in your predict.py file, and uses another popular Python package called FastAPI to dynamically build an OpenAPI schema and HTTP web server that can be used as a JSON API to generate predictions from your model. And because it's using OpenAPI, this server also dynamically generates documentation specific to your model, and a Swagger UI interface for trying out the API.

    With this new schema-focused design, the inputs and outputs you define for your model are converted into a portable, language-agnostic structured data format. This helps guide us toward a more standardized way of defining model interfaces and paves the way to better documentation, more automated tests, easier integrations across systems, and less reinventing the wheel.

    Installation

    First, install Docker if you haven't already. Then run this in a terminal:

    sudo curl -o /usr/local/bin/cog -L https://github.com/replicate/cog/releases/download/v0.1.0-alpha/cog_`uname -s`_`uname -m`
    sudo chmod +x /usr/local/bin/cog
    

    Upgrading from Cog 0.0.x to 0.1.x

    In Cog versions up to 0.0.20 you described inputs using annotations on your predict method. For example:

    @cog.input("image", type=Path, help="Image to enlarge")
    @cog.input("scale", type=float, default=1.5, help="Factor to scale image by")
    def predict(self, image, scale):
        ...
    

    From Cog 0.1.0 onwards, we've started using Pydantic to define input and output types. Rather than describing inputs using annotations, you now describe them with type hinting. Here's how you'd define the same inputs now:

    def predict(self,
        image: Path = Input(description="Image to enlarge"),
        scale: float = Input(description="Factor to scale image by", default=1.5)
    ) -> Path:
        ...
    

    The parameters that Input() takes are pretty similar to those @cog.input() used to take. Here are the differences:

    • It no longer takes a type parameter; use a type hint instead.
    • The help parameter has been renamed to description.
    • The options parameter has been renamed to choices.
    • The min option has been replaced by two options:
      • ge: greater than or equal to (direct replacement)
      • gt: greater than (a new alternative)
    • The max option has been replaced by two options:
      • le: less than or equal to (direct replacement)
      • lt: less than (a new alternative)

    The other major difference is that you now need to define the output type of your model. That's the -> Path bit in the example above. That might be a simple type like str, float or bool. If you need to handle multiple outputs, check out the new documentation for complex output objects. If you only have a single output, but it can be of different types depending on the run, you can use typing.Any:

    from typing import Any
    
    def predict(self) -> Any:
        ...
    

    Reference docs for Cog's new Python API can be found here.

    🆕 This is a prerelease version, so there are likely to be some rough edges. If you have any trouble please open an issue. :octocat:

    Changelog

    • ad918f9 Add cog.BaseModel
    • add4304 Add some more vscode settings from my local config
    • c9b0a84 Add support for redis queue worker
    • dacc30b Better error message when providing invalid inputs
    • f999872 Bump github.com/anaskhan96/soup from 1.2.4 to 1.2.5
    • 572c921 Bump github.com/docker/cli
    • 1ef44fa Bump github.com/docker/docker
    • 9c9659a Bump github.com/golangci/golangci-lint from 1.42.1 to 1.43.0
    • a29457d Bump github.com/golangci/golangci-lint from 1.43.0 to 1.44.0
    • a9e31c6 Bump github.com/golangci/golangci-lint from 1.44.0 to 1.44.1
    • 1333cbc Bump github.com/spf13/cobra from 1.2.1 to 1.3.0
    • 7c5d4b8 Bump pillow from 8.3.2 to 9.0.0
    • e01e07e Clean up build integration tests
    • c172527 Correctly pass environment variables to docker run
    • 944b5fc Display better error when output type is undefined
    • 7e91261 Don't require defining setup()
    • 6e71e31 Fix lint errors
    • 2b4f059 Implement choices option on input
    • 27c63bf Install wheel with pyenv Python
    • 79f440d Merge branch 'main' into future
    • ea3dac8 Merge pull request #420 from replicate/better-input-validation-messages
    • 44df62b Merge pull request #421 from replicate/throw-error-for-missing-output-type
    • c83030f Move integration test project to fixtures
    • 3158cb1 Move predict integration tests to fixtures
    • 94f466f Remove AI Platform server
    • cae046b Remove non-essential arguments to cog.Input()
    • 7e7e2b5 Remove old test fixtures
    • 12d5fbb Rename Predictor to BasePredictor
    • a62c4bc Rename annotation to InputType for clarity
    • 758a4bf Rename global.Verbose to global.Debug
    • 24112a3 Reorganize tests
    • 999756c Run Python tests on multiple Python versions
    • cda6c97 Run redis integration tests in Docker network
    • a73c1a7 Simplify tests by only testing setup() in one place
    • 6977af8 Skip predict with remote image test
    • 81bb5af Test complex output
    • 393ab98 Update documentation to use Pydantic annotations
    • 84d0bd5 Upgrade numpy to 1.21.5
    • d15a16d Upgrade to redis-py 4
    • e2e8f91 Use Pydantic for predictor type annotations
    • 9a1a828 add TOC to README
    • 42de67f add docstrings to new python functions
    • 43f9996 add pip install to CONTRIBUTING
    • 23be217 add prereqs to README
    • 06fb9c7 allow server log level to be configured with COG_LOG_LEVEL
    • 22b8fef always define components.schemas.Output
    • 33d74ba chore: support and document prereleases (#431)
    • d826854 clarify supported Python versions
    • 9467d76 document how to run tests
    • 631fe08 document new Python API
    • 76a8df2 nestle the story and shorten the TOC
    • 5a85615 recognize all contributors
    • c96c993 remove leftover prereq
    • e01b41a update VS Code settings to format on save
    • 68e04a7 update cog init predictor template
    • a4bc493 update prereqs to mention Python versions
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(9.23 MB)
    cog_Darwin_x86_64(9.42 MB)
    cog_Linux_arm64(8.12 MB)
    cog_Linux_x86_64(8.75 MB)
  • v0.0.19(Dec 15, 2021)

    Changelog

    • 1c72869 Add full example of cog.yaml to getting started
    • 9eb3d68 Bump github.com/docker/cli
    • f189ecc Bump github.com/docker/docker
    • adcfb4a Bump golang.org/x/tools from 0.1.7 to 0.1.8 (#351)
    • 2fa77d6 Configure VSCode for py.test
    • 00c8010 Fix typo in getting-started.md (#349)
    • c70fae5 Format
    • 3d4ae9f Mock time instead of sleeping in tests
    • 6fddf6e Run CI on all pushes
    • ec98b1c Support predict modules in subdirectories
    • 448d392 Throw error if invalid keys exist in cog.yaml (#325)
    • d98d4b2 Tiny - fix some typos in docs
    • cd92c18 Update email
    • dc95ed2 Update yaml.md - fix anchor links to subheaders
    • 40af690 chore: split python tests into multiple files
    • a293085 document how to automatically sign off commits
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(8.70 MB)
    cog_Darwin_x86_64(8.91 MB)
    cog_Linux_arm64(7.68 MB)
    cog_Linux_x86_64(8.28 MB)
  • v0.0.17(Nov 15, 2021)

  • v0.0.16(Nov 10, 2021)

    Changelog

    f9f59f0 Add golangci config to enable golint and other useful checks 5007f2d Fix Install Docker Link in Getting Started Docs 65d8dcd Remove tab 08c8aae Upgrade Torch/Tensorflow/CUDA compatibility maps ce41086 document prerequisites e60d81b document r8.im registry in YAML reference 38022a8 link to anchors 643da6e no trained model required for basic getting started guide 82f973e remove trained model prereq 4d55d41 use r8.im as the only image example

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(8.08 MB)
    cog_Darwin_x86_64(8.28 MB)
    cog_Linux_arm64(7.18 MB)
    cog_Linux_x86_64(7.73 MB)
  • v0.0.15(Oct 19, 2021)

  • v0.0.14(Oct 5, 2021)

    Changelog

    843294f Add Docker install to readme a52ff80 Add install cog instructions to getting started 29fe0e6 Bump golang.org/x/tools from 0.1.6 to 0.1.7 8c296dc Check if remote image exists before pulling in predict c06a01d Debug statement in ImageInspect 063d0c0 Fix formatting adf3295 Make remote models use GPUs in cog predict, if model requires GPU 76c3712 Relax torchvision version check 3741f20 Use r8.im in docs 290627d feat: add new command cog init (#279) 37b570c minor tweak to function names in example 9346ce7 note that build runs before first prediction f0a9fc7 update cog.yaml reference doc c7e038a use curl instead of wget to download weights (#271)

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(8.08 MB)
    cog_Darwin_x86_64(8.27 MB)
    cog_Linux_arm64(7.18 MB)
    cog_Linux_x86_64(7.72 MB)
  • v0.0.13(Sep 28, 2021)

  • v0.0.12(Sep 22, 2021)

    Changelog

    2f70db8 Add "failed" status to redis responses ce1de79 Add support for yield to redis queue worker 3770198 Bump golang.org/x/tools from 0.1.5 to 0.1.6 18e8fa4 Capture log output from generator bcb9b44 Create extensions.json e984c84 Document not downloading weights in setup() b048b49 Set timeouts for brpop tests 78df85d [wip] Yield progressive output 7017729 clarify installation steps for Mac M1 users 4f2c573 clean up the wheel 13d874d document release process (#236) 0ca463e make fmt 1ab99b2 refine M1 copy 7180bcf update http responder to handle generator functions fdc2804 use sudo to install

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(8.06 MB)
    cog_Darwin_x86_64(8.26 MB)
    cog_Linux_arm64(7.18 MB)
    cog_Linux_x86_64(7.71 MB)
  • v0.0.11(Sep 13, 2021)

    Changelog

    66b44eb Allow PyTorch GPU versions not known by Cog 4dedf07 Better build temp directory ddfcecf Bump github.com/golangci/golangci-lint from 1.42.0 to 1.42.1 b9aef0b Bump github.com/mattn/go-isatty from 0.0.13 to 0.0.14 9a4bcf9 Bump pillow from 8.2.0 to 8.3.2 (#234) edd4063 More sensible pytest output 9d84df8 Remove main-trigger from CI 39cfc5c add troubleshooting section 975e2b2 link to golang install page

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(332 bytes)
    cog_Darwin_arm64(8.06 MB)
    cog_Darwin_x86_64(8.26 MB)
    cog_Linux_arm64(7.18 MB)
    cog_Linux_x86_64(7.70 MB)
  • v0.0.10(Sep 3, 2021)

Owner
Replicate
Making machine learning reproducible
Replicate
Machine Learning for Go

GoLearn GoLearn is a 'batteries included' machine learning library for Go. Simplicity, paired with customisability, is the goal. We are in active deve

Stephen Whitworth 8.4k Jun 25, 2022
On-line Machine Learning in Go (and so much more)

goml Golang Machine Learning, On The Wire goml is a machine learning library written entirely in Golang which lets the average developer include machi

Conner DiPaolo 1.3k Jun 30, 2022
Gorgonia is a library that helps facilitate machine learning in Go.

Gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily

Gorgonia 4.5k Jun 26, 2022
Machine Learning libraries for Go Lang - Linear regression, Logistic regression, etc.

package ml - Machine Learning Libraries ###import "github.com/alonsovidales/go_ml" Package ml provides some implementations of usefull machine learnin

Alonso Vidales 193 Jun 23, 2022
Gorgonia is a library that helps facilitate machine learning in Go.

Gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily

Gorgonia 4.5k Jun 26, 2022
Prophecis is a one-stop machine learning platform developed by WeBank

Prophecis is a one-stop machine learning platform developed by WeBank. It integrates multiple open-source machine learning frameworks, has the multi tenant management capability of machine learning compute cluster, and provides full stack container deployment and management services for production environment.

WeBankFinTech 344 Jun 22, 2022
Go Machine Learning Benchmarks

Benchmarks of machine learning inference for Go

Nikolay Dubina 23 May 27, 2022
A High-level Machine Learning Library for Go

Overview Goro is a high-level machine learning library for Go built on Gorgonia. It aims to have the same feel as Keras. Usage import ( . "github.

AUNUM 300 Jun 16, 2022
Katib is a Kubernetes-native project for automated machine learning (AutoML).

Katib is a Kubernetes-native project for automated machine learning (AutoML). Katib supports Hyperparameter Tuning, Early Stopping and Neural Architec

Kubeflow 1.2k Jun 23, 2022
PaddleDTX is a solution that focused on distributed machine learning technology based on decentralized storage.

中文 | English PaddleDTX PaddleDTX is a solution that focused on distributed machine learning technology based on decentralized storage. It solves the d

null 58 Jun 23, 2022
Self-contained Machine Learning and Natural Language Processing library in Go

Self-contained Machine Learning and Natural Language Processing library in Go

NLP Odyssey 1.2k Jun 24, 2022
Example of Neural Network models of social and personality psychology phenomena

SocialNN Example of Neural Network models of social and personality psychology phenomena This repository gathers a collection of neural network models

null 6 Jan 15, 2022
Reinforcement Learning in Go

Overview Gold is a reinforcement learning library for Go. It provides a set of agents that can be used to solve challenges in various environments. Th

AUNUM 272 Jun 29, 2022
Spice.ai is an open source, portable runtime for training and using deep learning on time series data.

Spice.ai Spice.ai is an open source, portable runtime for training and using deep learning on time series data. ⚠️ DEVELOPER PREVIEW ONLY Spice.ai is

Spice.ai 738 Jun 21, 2022
FlyML perfomant real time mashine learning libraryes in Go

FlyML perfomant real time mashine learning libraryes in Go simple & perfomant logistic regression (~100 LoC) Status: WIP! Validated on mushrooms datas

Vadim Kulibaba 1 May 30, 2022
Go (Golang) encrypted deep learning library; Fully homomorphic encryption over neural network graphs

DC DarkLantern A lantern is a portable case that protects light, A dark lantern is one who's light can be hidden at will. DC DarkLantern is a golang i

Raven 1 Dec 2, 2021
A tool for building identical machine images for multiple platforms from a single source configuration

Packer Packer is a tool for building identical machine images for multiple platforms from a single source configuration. Packer is lightweight, runs o

null 2 Oct 3, 2021
Deploy, manage, and scale machine learning models in production

Deploy, manage, and scale machine learning models in production. Cortex is a cloud native model serving platform for machine learning engineering teams.

Cortex Labs 7.8k Jun 21, 2022
This is the repository for the LinkedIn Learning course Learning Go.

Learning Go This is the repository for the LinkedIn Learning course Learning Go. The full course is available from LinkedIn Learning. What is Go? Go i

Zhenguan Tang 0 Nov 2, 2021
This repository is where I'm learning to write a CLI using Go, while learning Go, and experimenting with Docker containers and APIs.

CLI Project This repository contains a CLI project that I've been working on for a while. It's a simple project that I've been utilizing to learn Go,

Tamir Arnesty 0 Dec 12, 2021
Selected Machine Learning algorithms for natural language processing and semantic analysis in Golang

Natural Language Processing Implementations of selected machine learning algorithms for natural language processing in golang. The primary focus for t

James Bowman 367 Jun 25, 2022
Machine Learning for Go

GoLearn GoLearn is a 'batteries included' machine learning library for Go. Simplicity, paired with customisability, is the goal. We are in active deve

Stephen Whitworth 8.4k Jun 25, 2022
On-line Machine Learning in Go (and so much more)

goml Golang Machine Learning, On The Wire goml is a machine learning library written entirely in Golang which lets the average developer include machi

Conner DiPaolo 1.3k Jun 30, 2022
Gorgonia is a library that helps facilitate machine learning in Go.

Gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily

Gorgonia 4.5k Jun 26, 2022
Machine Learning libraries for Go Lang - Linear regression, Logistic regression, etc.

package ml - Machine Learning Libraries ###import "github.com/alonsovidales/go_ml" Package ml provides some implementations of usefull machine learnin

Alonso Vidales 193 Jun 23, 2022
Gorgonia is a library that helps facilitate machine learning in Go.

Gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily

Gorgonia 4.5k Jun 26, 2022
Sparse matrix formats for linear algebra supporting scientific and machine learning applications

Sparse matrix formats Implementations of selected sparse matrix formats for linear algebra supporting scientific and machine learning applications. Co

James Bowman 131 Jun 27, 2022
Selected Machine Learning algorithms for natural language processing and semantic analysis in Golang

Natural Language Processing Implementations of selected machine learning algorithms for natural language processing in golang. The primary focus for t

James Bowman 366 Jun 15, 2022
Self-contained Machine Learning and Natural Language Processing library in Go

If you like the project, please ★ star this repository to show your support! ?? A Machine Learning library written in pure Go designed to support rele

NLP Odyssey 1.2k Jun 27, 2022