Hermit - uniform tooling for Linux and Mac

Related tags

Command Line hermit
Overview
Hermit

Hermit - uniform tooling for Linux and Mac CI Slack chat

Hermit installs tools for software projects in self-contained, isolated sets, so your team, your contributors, and your CI have the same consistent tooling.

See https://cashapp.github.io/hermit for full documentation.

hermit intro screen cast


Copyright 2021 Square, Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Comments
  • Shell hooks are not installed when running `zsh --login`

    Shell hooks are not installed when running `zsh --login`

    I found this issue working with Hermit in VS Code. When we launch VS Code from a hermit-activated environment/shell, it fails to set the PATH correctly for the underlying terminal:

    source ./bin/activate-hermit
    echo $HERMIT_ENV                       # /path/to/my/repo
    which python3                          # /path/to/my/repo/bin/python3
    

    Which tells me that the environment is activated successfully, however, if I open a VS Code instance from that terminal:

    code /path/to/my/repo
    

    And try to use the integrated terminal:

    echo $HERMIT_ENV                       # /path/to/my/repo
    which python3                          # /usr/bin/python3
    

    Which means that HERMIT_ENV is propagated, but PATH is not propagated correctly. To fix this, I have to activate it again from the integrated terminal, even though it is using zsh, which has the right hermit shell-hooks in .zshrc:

    source ./bin/activate-hermit
    
    This Hermit environment has already been activated. Skipping
    

    I then get this confusing message, however, the shell is configured correctly now:

    which python3                          # /path/to/my/repo/bin/python3
    

    I wonder if there is a better way to detect this better than using HERMIT_ENV. We can either:

    1. fix this for integrated terminals, so that PATH is propagated correctly.
    2. Detect that the shell is not truly configured, and update error message accourdingly.
    opened by OmarTawfik 17
  • split commands.go into multiple files

    split commands.go into multiple files

    commands.go was looking increasingly large. I broke it into separate files and it's own package to keep things tidy.

    Note that some small commands (noop, version) I left behind just because they were small and more... operational).

    Also a caveat is that app/commands/ does not have access to Config, so I had to work around that by binding the http client and the env vars. I'm open to better ideas there..

    Added a little something to the makefile, in case it eventually gets bigger. Looks like this: Screen Shot 2021-09-16 at 5 39 44 PM )

    opened by mihai-chiorean 13
  • Lock the digests by adding sha256 values for installed packages.

    Lock the digests by adding sha256 values for installed packages.

    Populate sha256sums in a specified manifest file Usage: hermit manifest add-digests <manifest.hcl>

    It should populate(in place) all the sha256sums for all the version/platform combinations in the original manifest file.

    Testing:

    • An integration test has been added to verify the digest created.
    • An integration test has been added to verify that hermit install fails on wrong digest.
    opened by nmahendru 7
  • On mac with zsh, you need gmake so that the

    On mac with zsh, you need gmake so that the "source" command works in your makefiles

    on a mac with zsh you need gmake ( not make ) , so that source command works in your makefiles.

    maybe worth adding this to https://cashapp.github.io/hermit/usage/get-started/#initialising-a-project

    then the init works:

    	# add to zsh so the hermit hooks can fire.
    	hermit shell-hooks --zsh
    
    	# create a local bin with the hermit stuff.
    	hermit init
    
    	# loads the env.
    	source ./bin/activate-hermit
    
    opened by gedw99 6
  •  Permission denied issue with cache

    Permission denied issue with cache

    hermit install python3-3.9.11
    pip3 install poetry==1.2.0b1
    poetry install
    

    Poetry appears to be having a problem with all packages with the cache permissions.

    Installing collected packages: setuptools
        Attempting uninstall: setuptools
          Found existing installation: setuptools 60.9.3
          Uninstalling setuptools-60.9.3:
      ERROR: Could not install packages due to an OSError: Cannot move the non-empty directory '/Users/damien/Library/Caches/hermit/pkg/python3-3.9.11/install/lib/python3.9/site-packages/_distutils_hack/': Lacking write permission to '/Users/damien/Library/Caches/hermit/pkg/python3-3.9.11/install/lib/python3.9/site-packages/_distutils_hack/'.
    
    opened by damienrj 6
  • Support running hermit from different environments

    Support running hermit from different environments

    • [x] ๐Ÿ“ˆ Improve the comments
    • [x] โŒ Remove the special case in exec_cmd.go
    • [x] ๐Ÿ Check for bin/hermit instead of bin/hermit.hcl
    • [x] โ“ Move bin/hermit check into env
    opened by quad 5
  • Verify installer script checksum when bootstrapping

    Verify installer script checksum when bootstrapping

    We verify the downloaded install script's SHA-256 value against a known checksum in bin/hermit. This ensures the integrity of the installer when bootstrapping hermit.

    We version (via file name suffix) the install script by the script's short hash value, so that an install script can be uniquely identified by its version / file name. The corresponding bin/hermit is generated with hermit init for a project.

    Note that an install script's version" is unrelated to the version of the hermit executable.

    opened by syncom 5
  • fix: use the right installer URL for the canary channel

    fix: use the right installer URL for the canary channel

    • We make changes for the generated bin/hermit to correctly identity the download URL for the canary channel. This part was overlooked in an earlier implementation.
    • The InstallScriptSHAs map is manually populated for the "canary" and "stable" channels, respectively, in env.go.
    • Update ScriptSHAs for the updated bin/hermit scripts, for "canary" and "stable" channels.
    • Add InstallerSHA256Sums map to the Config struct of the main Hermit app.
    • Update GHA CI step for sanity check of SHA256 sums of install script.
    • Fix a buggy integration test.
    opened by syncom 5
  • Hermit shell hooks emits error

    Hermit shell hooks emits error

    When I use hermit's shellhooks in zsh, in a hermit enabled repo, I get the following error at the end of every single command I run.

    fatal:hermit: /Users/yunchi/Projects/bork/bin/hermit has an unknown SHA256 signature (9413f2347c5f70e6a004e62b7faac99d3bb1666f86451ed1f6e05a679e3bc27c); verify that you trust this environment and run 'hermit init /Users/yunchi/Projects/bork'
    

    This doesn't happen if I disable the shell hook and activate hermit manually.

    opened by mightyguava 5
  • Fix infinite ETag check loop if DB is missing.

    Fix infinite ETag check loop if DB is missing.

    If the hermit DB is deleted, executing binaries from any existing packages would result in an ETag check on each invocation.

    I think this behaviour is okay, but it's also not clear to me why the original behaviour was desirable?

    opened by alecthomas 5
  • feat: add data uri scheme encoding as valid source

    feat: add data uri scheme encoding as valid source

    Proposed Solution (2.b.) from https://github.com/cashapp/hermit/issues/310.

    Example Usage

    Without additional sources:

    example_config=$(cat <<EOF
    description = "Description for example"
    binaries = ["example"]
    channel "unstable" {
     update = "5m"
     source = "[email protected]:cashapp/example.git#$EXAMPLE_VERSION"
    }
    EOF
    )
    
    override_sources=$(cat <<EOF
    {"example": $(echo "$example_config" | jq -R -s '.')}
    EOF
    )
    
    override_uri="data:application/json;base64,$(echo "$override_sources" | base64)"
    
    HERMIT_ADDITIONAL_SOURCES="$override_uri" hermit ...
    

    Example URI

    > echo "$override_uri"
    data:application/json;base64,eyJleGFtcGxlIjogImRlc2NyaXB0aW9uID0gXCJEZXNjcmlwdGlvbiBmb3IgZXhhbXBsZVwiCmJpbmFyaWVzID0gW1wiZXhhbXBsZVwiXQpjaGFubmVsIFwidW5zdGFibGVcIiB7CiB1cGRhdGUgPSBcIjVtXCIKIHNvdXJjZSA9IFwiZ2l0QGdpdGh1Yi5jb206Y2FzaGFwcC9leGFtcGxlLmdpdCNtYWluXCIKfQoifQo
    
    opened by MatthewDolan 4
  • nats object store

    nats object store

    I expect this is a little off field but will give it a try

    https://docs.nats.io/nats-concepts/jetstream/obj_store

    why ?

    • Store the binaries inside nats. so no github repo bloat.
    • leafs nodes are the perfect analogy to a offline setup.
    • AAA security is backed in with JWT and highly extensible.
    • scales infinity and is clustered.

    The reasons i thought of this is because NAST is a pub sub model, and Hermit is wanted to subscribe to change to the upstream repos. SO its really a similar thing. NATS can subscribe to releases from upstreams and then see the release, and pull it and consumes can have it magically be available for their Hermit systems and developers.

    opened by gedw99 0
  • Update `hermit` cli version from Renovate bot

    Update `hermit` cli version from Renovate bot

    Follow up from #342

    The current Renovate plugin supports updating versions of the binaries, but not the version of the hermit binary itself.

    This is usually specified in the bin/hermit file, and it defaults to using the stable channel:

    export HERMIT_DIST_URL="${HERMIT_DIST_URL:-https://github.com/cashapp/hermit/releases/download/stable}"
    

    This means projects using Hermit can and will be broken if a bad version of Hermit gets released to Stable channel.

    One solution would be to set a custom env var HERMIT_DIST_URL like https://github.com/cashapp/hermit/releases/download/v0.31.1. But that has two problems:

    1. This variable needs to be set before Hermit is actually invoked and loaded, which means that we need an external way to set it, other than the supported/recommended way in hermit.hcl env property.
    2. That will quickly go out of sync, and users need to update it manually regularly.

    I suggest that if a repo locks the default installation URL in bin/hermit to specific versions, that Renovate bot updates it during regular updates. This means:

    1. Users will stay up to date.
    2. No need for an external (additional to Hermit) solution to manage environment variables before Hermit gets loaded.
    3. This change is backwards compatible. users can still stay on stable channel if they want, and the bot won't change it if so.
    opened by OmarTawfik 3
  • Update `sources` URL refs from Renovate Bot

    Update `sources` URL refs from Renovate Bot

    Follow up from https://github.com/cashapp/hermit/issues/342

    Users can use a specific git reference to fix their package sources in hermit.hcl sources property. Example:

    # hermit.hcl
    sources = [
      "https://github.com/cashapp/hermit-packages.git#ref"
    ]
    

    This means it will quickly get out of date, and it needs updating regularly.

    I suggest adding support for this in the Renovate bot, by updating the ref suffix part of the URL regularly: https://docs.renovatebot.com/modules/manager/hermit/

    Note: for most use cases, this means fetching the latest commit from the default branch of the repository they are using (origin/HEAD), but for completeness, it can support specifying a custom remote reference (like refs/HEAD/branch-name) for users to put into their renovate.json config, for repositories that use a separate release branch.

    opened by OmarTawfik 0
  • feat: Git manifest sources can now specify a reference to pin to

    feat: Git manifest sources can now specify a reference to pin to

    This was in the documentation but not implemented (as discovered in #342). #<tag> was supported for Git packages, so I've factored that code out and reused it.

    Added an integration test for this, along with a couple of new test helper functions that should make these kind of tests simpler in the future.

    opened by alecthomas 2
  • source urls cannot be versioned

    source urls cannot be versioned

    A recent update in hermit-packages broke all rust pipelines: https://github.com/cashapp/hermit-packages/issues/235

    To avoid this, projects typically fix their dependencies versions. As Hermit docs suggest, this can be done by locking the version of the packages sources they import.

    An optional # suffix can be added to checkout a specific tag.

    Which suggests something like this:

    # hermit.hcl
    sources = [
      "https://github.com/cashapp/hermit-packages.git#tag"
    ]
    

    However, I had the following issues using that feature:

    One: hermit-packages does not push tags regularly

    For users to stay up to date, there need to be regular tags/versions pushed from this repository. Looking at the existing tags, I find only one index tag that is 6 months old: https://github.com/cashapp/hermit-packages/tags

    If pushing tags more regularly is useful for Hermit long-term plans, should it be automated? Otherwise, I suggest supporting any ref in the URL, so that users can specify a specific commit hash (instead of a tag), and update it regularly.

    Two: url format is not clear

    Now I tried using the following formats, which all failed:

    • https://github.com/cashapp/hermit-packages.git#index
      • fatal:hermit: /github/workspace/bin/hermit.hcl: unsupported source
    • https://github.com/cashapp/hermit-packages#index.git
      • exit status 128: git sync failed: git clone --depth=1 failed

    I suggest adding an explicit example of a versioned url in sources array in the documentation, for clarity.

    Three: regular updates

    Looking at Renovate plugin docs, it is not clear to me if the bot can update tags (or commit hashes) in the sources URLs in hermit.hcl .. Is this a supported scenario?

    opened by OmarTawfik 6
  • feat: Adding RequiredDigests

    feat: Adding RequiredDigests

    This forces every package that is about to be cached to have a digest to check against.

    This is a building block for enforcing digest checks if we want to.

    opened by nmahendru 0
Releases(v0.32.0)
CLI to support with downloading and compiling terraform providers for Mac with M1 chip

m1-terraform-provider-helper A CLI to help with managing the installation and compilation of terraform providers when running a new M1 Mac. Motivation

kreuzwerker GmbH 265 Jan 2, 2023
Termial remote (mac) for Sony Bravia TV

Sony Bravia terminal remote Environment variables HOST: [....] PSK: [....] Configure the Bravia TV Todo: Scan network for a Bravia device Move and git

Emil 1 Nov 26, 2021
Service for read & update MAC OUI list

manufacturer-parser Service written on Golang to get actual MAC OUI list from here and store to MongoDB. Needed for dealt with actual list in Wimark O

WiMark Systems 0 Dec 29, 2021
Oui - MAC Address CLI Toolkit

oui MAC Address CLI Toolkit Installation macOS $ brew tap thatmattlove/oui $ bre

Matt Love 26 Jan 1, 2023
A twitch focused command line tool for producing, archiving and managing live stream content. Built for Linux.

twinx is a live-streaming command line tool for Linux. It connects streaming services (like Twitch, OBS and YouTube) together via a common title and description.

Kris Nรณva 26 Oct 17, 2022
Command-line tool to customize the official Spotify client. Supports Windows, MacOS and Linux.

Command-line tool to customize the official Spotify client. Supports Windows, MacOS and Linux. Features Change colors whole UI Inject CSS for advanced

null 13.1k Jan 2, 2023
Runc: a CLI tool for spawning and running containers on Linux according to the OCI specification

runc Introduction runc is a CLI tool for spawning and running containers on Linux according to the OCI specification. This repo contains a lightly mod

Brian 0 Dec 16, 2021
Alpie - A CLI tool for imaging and configuring Alpine Linux installs on Raspberry Pis

Alpie Alpie is a CLI tool aimed at making imaging and customizing installs of Al

George Bolmida 1 Jan 22, 2022
Alpie - A CLI tool for imaging and configuring Alpine Linux installs on Raspberry Pis

Alpie is a CLI tool aimed at making imaging and customizing installs of Alpine Linux onto Raspberry Pis simpler.

George Bolmida 1 Jan 22, 2022
progress_bar creates a single customizable progress bar for Linux terminal.

progress_bar Go Progress Bar Features progress_bar creates a single customizable progress bar for Linux terminal. Installation go get -u github.com/er

erman imer 127 Aug 12, 2022
gif effects CLI. single binary, no dependencies. linux, osx, windows.

yeetgif Composable GIF effects CLI, with reasonable defaults. Made for custom Slack/Discord emoji :) Get it Alternative 1: go get Alternative 2: just

Sergey Grebenshchikov 530 Dec 11, 2022
1Pass - 1Password Linux CLI explorer

1Pass is a command line application that allows to explore 1Password OPVault format. Application was created because there is no official 1Password desktop client for Linux users.

Maciej Bedra 8 Nov 14, 2022
Go package for running Linux distributed shell commands via SSH.

Go package for running Linux distributed shell commands via SSH.

Disco 148 Dec 7, 2022
A tool to enumerate all the command-line arguments used to start a Linux process written in Go.

ranwith A tool to enumerate all the command-line arguments used to start a Linux process written in Go. ranwith uses the Linux /proc directory to obta

Alexis Rodriguez 3 Jun 30, 2022
MimixBox - mimic BusyBox on Linux

MimixBox - mimic BusyBox on Linux MimixBox has many Unix commands in the single binary like BusyBox. However, mimixbox aim for the different uses from

Nao1215 24 Nov 30, 2022
Tool for containerized command line environments on Linux

Tool for containerized command line environments on Linux

Containers 1.6k Dec 31, 2022
ets2-dlc-repacker is a Windows / Linux / MacOS CLI util to automatically repack older DLC archives for compatibility with newer versions.

ets2-dlc-repacker is a Windows / Linux / MacOS CLI util to automatically repack older DLC archives for compatibility with newer versions.

Daniel 0 Dec 26, 2021
Clipsync: a high performance clipboard sharing application for linux

clipsync: Clipboard sync Clipsync is a high performance clipboard sharing applic

Zachary Huff 3 Sep 18, 2022
Jsos - A operating system that runs system-level javascript, based on the Linux kernel

JsOS ?? An linux-based operating system that runs Javascript code at the system-

Theo Paris 2 Jan 6, 2023