Database security suite. Database proxy with field-level encryption, search through encrypted data, SQL injections prevention, intrusion detection, honeypots. Supports client-side and proxy-side ("transparent") encryption. SQL, NoSQL.

Overview

Acra: database security suite
Database protection suite with field level encryption and intrusion detection.


GitHub release Circle CI Coverage Status godoc
Server platforms Client platforms Install on DigitalOcean


| Acra Engineering Examples | Documentation | Installation | Examples and tutorials | | ---- | ---- | ---- | --- | --- |

What is Acra

Acra — database security suite for sensitive and personal data protection.

Acra provides field level encryption (client-side or proxy-side), multi-layered access control, database leakage prevention, and intrusion detection capabilities in a convenient, developer-friendly package. Acra was specifically designed for web and mobile apps with centralised data storage, including with distributed, microservice-rich applications.

Perfect Acra-compatible applications Typical industries
Web and mobile apps that store data in a centralised database or object storage
  • Healthcare
  • Finance
  • E-commerce
  • Critical infrastructures
  • Apps with > 1000 users
IoT apps that collect telemetry and process data in the cloud
High-load data processing apps

Acra gives you tools for encrypting the data records into special cryptographic containers, storing them in the database or file storage and then decrypting them in a secure compartmented area (separate virtual machine/container). Acra allows to encrypt data as early as possible and operate on encrypted data.

Acra's cryptographic design ensures that no secret (password, key, etc.) leaked from the application or database will be sufficient for decryption of the protected data chunks that originate from it. Acra minimises the leakage scope, detects unauthorised behavior, and prevents the leakage, informing operators of the incident underway.

This is Acra Community Edition, it's free for commercial and non-commercial use, forever.

User Acra suite for protecting data

Major security features

  • Cryptographic protection of data
  • field level encryption (client-side and/or proxy-side);
    each data record is encrypted using unique encryption keys
  • Selective encryption
  • select which field/columns to encrypt to balance good security and performance
  • Key management tools
  • built-in tools for key distribution, key rotation (with/without data re-encryption), and compartmentalisation
  • Trust compartmentalisation
  • datastore and application components can be compromised, yet the data stays protected
  • Prevention of SQL injections
  • through a built-in SQL firewall
  • Intrusion detection system
  • using poison records (honey tokens) to warn about suspicious behaviour
  • REST API (gRPC, HTTP)
  • encryption/decryption using AcraTranslator
  • Searchable encryption ᵉ
  • available for Acra Enterprise users
  • Masking / anonymization ᵉ
  • Tokenization / Pseudonymization ᵉ
  • Cryptographically protected audit log ᵉ
  • Acra delivers different layers of defense for different parts and stages of the data lifecycle. This is what defence in depth is – an independent set of security controls aimed at mitigating multiple risks in case of an attacker crossing the outer perimeter.

    Developer and DevOps friendly

  • Secure default settings
  • your infrastructure is secure from the start without additional configuring
  • Cryptography is hidden
    under the hood
  • no risk of selecting the wrong key length or algorithm padding
  • Automation-friendly
  • easy to configure and automate
  • Quick infrastructure integration
  • via binary packages or Docker images
  • Easy client-side integration
  • client-side encryption libraries support ~11 languages
  • Code-less client-side integration
  • available for server-side encryption in AcraServer's Transparent proxy mode
  • Logging, metrics, tracing
  • throughout all Acra components;
    compatible with ELK stack, Prometheus, Jaeger
  • No vendor lock
  • rollback utilities to decrypt database into plaintext
  • Demos, examples, simulators
  • numerous web-based and Docker-based example projects available
  • DigitalOcean Acra 1-Click App
  • run AcraServer in your DigitalOcean cloud
  • Managed solution available
  • we can setup and manage Acra for you

    Cryptography

    Acra relies on our cryptographic library Themis, which implements high-level cryptosystems based on the best available open-source implementations of the most reliable ciphers. Acra strictly doesn't contain self-made cryptographic primitives or obscure ciphers. To deliver its unique guarantees, Acra relies on the combination of well-known ciphers and smart key management scheme.

    Default crypto-primitive source OpenSSL
    Supported crypto-primitive sources ᵉ BoringSSL, LibreSSL, FIPS-compliant, GOST-compliant, HSM
    Storage encryption AES-256-GCM + ECDH
    Transport encryption TLS v1.2+ / Themis Secure Session
    KMS integration ᵉ Amazon KMS, Google Cloud Platform KMS, Hashicorp Vault, Keywhiz

    ᵉ — available in the Enterprise version of Acra only. Drop us an email to get a full list of features and a quote.

    Try Acra without writing code

    How does Acra work?

    To better understand the architecture and data flow in Acra, please refer to the Architecture and data flow section in the documentation.

    Protecting data in SQL databases using AcraServer

    AcraServer works as transparent encryption/decryption proxy with SQL databases. Depending on your app architecture, it's possible to use client-side encryption or server-side encryption, or both simultaneously.

    Integrating client-side encryption using AcraWriter

    Client-side encryption using AcraServer and AcraWriter

    The typical infrastructure with AcraConnector and AcraServer, data is encrypted on the client side (web backend or mobile app) using AcraWriter, decrypted on the AcraServer side, that works as transparent database proxy.

    Client-side encryption using AcraServer and AcraWriter

    The dataflow of encrypted and plaintext data. Note: it's possible to write encrypted data directly to the database, omitting the AcraServer.

    This is what the process of encryption and decryption of data in a database looks like:

    • Your application encrypts some data through AcraWriter by generating an AcraStruct using Acra storage public key and then updates the database. AcraStructs generated by AcraWriter can't be decrypted by it — only the Acra's server side has the keys for decryption.
    • To retrieve the decrypted data, your application talks to AcraServer. It is a server-side service that works as a database proxy: it transparently sits between your application and the database and silently listens to all the traffic coming to and from the database.
    • AcraServer monitors the incoming SQL requests and blocks the unwanted ones using the built-in configurable SQL firewall called AcraCensor. AcraServer only sends allowed requests to the database. Certain configurations for AcraServer can be adjusted remotely using AcraWebConfig web server.
    • Upon receiving the database response, AcraServer tries to detect AcraStructs, decrypts them, and returns the decrypted data to the application.
    • AcraConnector is a client-side daemon responsible for providing encrypted and authenticated connection between the application and AcraServer. AcraConnector runs under a separate user/in a separate container and acts as middleware. AcraConnector accepts connections from the application, adds an extra transport encryption layer using TLS or Themis Secure Session, sends the data to AcraServer, receives the result, and sends it back to the application.

    Integrating proxy-side encryption using AcraServer in Transparent proxy mode

    Server-side encryption using AcraServer

    The typical infrastructure with AcraConnector and AcraServer, data is encrypted and decrypted by AcraServer, that works as transparent database proxy.

    Server-side encryption using AcraServer

    The dataflow of encrypted and plaintext data, AcraServer it encrypts and decrypts data.

    Transparent proxy mode allows you to configure AcraServer to encrypt records in specific database columns without altering the application code. Basically, AcraServer here performs AcraWriter's duties.

    This mode is useful for large distributed applications, where separate updating of the source code for each client app is complicated.

    • Imagine that the client application wants to update some data in the database. The application logic doesn't need to change because there's no need to integrate AcraWriter library into the application code. The application sends SQL requests with plaintext data through AcraConnector and AcraServer to the database, as described above. There's a secure transport layer between AcraConnector and AcraServer (via TLS or Themis Secure Session), so data from client app is transmitted via a secure channel.
    • On receiving SQL queries, AcraServer parses each query and encrypts the desired values into AcraStructs. To know which values to encrypt, AcraServer uses a special configuration file (known as encryptor_config_file) that describes which columns in which tables should be encrypted. AcraServer uses AcraStruct encryption key just as AcraWriter would.
    • After encrypting the values into AcraStructs, AcraServer passes the modified queries to the database and the database response – back to the client application via AcraConnector.
    • When the client application wants to read the data, it sends a read query to the database (via AcraConnector and AcraServer). Upon retrieving the database response, AcraServer tries to detect AcraStructs, decrypts them, and returns the decrypted data to the application.

    Transparent mode has the same features as the "usual" mode – including SQL firewall, intrusion detection based on poison records, Zone support, etc.

    Check out the detailed documentation on using and configuring the Transparent proxy mode.

    Protecting data in any file storage using AcraWriter and AcraTranslator

    Client-side encryption and standalone decryption using AcraTranslator

    The typical infrastructure with AcraConnector and AcraTranslator, data is encrypted on the client side (web backend or mobile app) using AcraWriter, decrypted on the AcraTranslator side, that works as standalone decryption service.

    Acra Translator: simplified architecture

    The dataflow of encrypted and plaintext data, AcraTranslator only decrypts data on demand.

    In some use cases, the application can store encrypted data as separate blobs (in NoSQL databases, or as files that are not in a database, i.e. in a S3 bucket, local file storage, etc.). In this case, you can use AcraTranslator — a lightweight server that receives AcraStructs and returns the decrypted data.

    Note: since Q3 2020, AcraTranslator can both encrypt and decrypt data via REST API (HTTP, gRPC). You may use a combination of AcraWriter+AcraTranslator for client-side encryption and Translator-side decryption; or AcraTranslator only for both encryption and decryption without changing app code.

    This is what the process of encryption and decryption of data using AcraTranslator looks like:

    • Your application encrypts some data using AcraWriter, generating an AcraStruct using Acra storage public key and puts the data into any file storage. AcraStructs generated by AcraWriter can't be decrypted by it — only the Acra's server side has the right keys for decrypting it. Otherwise, you might decide not to use AcraWriter, but use AcraTranslator for encrypting data – application sends plaintext, AcraTranslator returns AcraStructs.
    • To decrypt an AcraStruct, your application sends it to AcraTranslator as a binary blob via HTTP or gRPC API. AcraTranslator doesn’t care about the source of the data, it is responsible for holding all the secrets required for data decryption and for actually decrypting the data.
    • AcraTranslator decrypts AcraStructs and returns the decrypted data to the application.
    • To avoid sending plaintext via an unsecured channel, AcraTranslator requires the use of AcraConnector, a client-side daemon responsible for providing encrypted and authenticated connection between the application and AcraServer. AcraConnector runs under a separate user/in a separate container and acts as middleware. It accepts connections from the application, adds transport encryption layer using TLS or Themis Secure Session, sends data to AcraServer, receives the result, and sends it back to the application.

    AcraTranslator and AcraServer are fully independent server-side components and can be used together or separately depending on your infrastructure.

    Availability

    Client-side

    AcraWriter is a client-side library that encrypts data into a special binary format called AcraStruct. AcraWriter is available for Ruby, Python, Go, C++, Node.js, iOS, Android/Java and PHP, but you can easily generate AcraStruct containers with Themis for any platform you want.

    AcraWriter is required only for the client-side encryption dataflow. If you use AcraServer in Transparent encryption mode, or use AcraTranslator for both encryption and decryption, you don't need AcraWriter at all.

    Client platform Documentation and guides Examples Package manager
    🐹 Go Installation guide examples/golang
    🐍 Python Installation guide examples/python PyPI
    ♦️ Ruby Installation guide examples/ruby Gem
    C++ Installation guide examples/cpp
    📱 iOS (Swift / Objective-C) Installation guide examples/objc CocoaPods
    ☎️ Android (Java / Kotlin) Installation guide examples/android_java maven
    🐘 PHP Installation guide examples/php
    🍭 Node.js Installation guide examples/nodejs npm

    Server-side

    The Server-side Acra components (AcraServer, AcraTranslator) should run as separate services/servers/VMs. Key management utils and database helpers are usually run on the same server as AcraServer/AcraTranslator.

    Install Server-side Acra components:

    Distributive Instruction set Download package or alternatively
    CentOS 7 x86_64 rpm package use Docker or
    build from source
    Debian Stretch (9)
    Debian Jessie (8)
    x86_64/i386 deb package
    Ubuntu Bionic (18.04) x86_64 deb package
    Ubuntu Artful (17.10)
    Ubuntu Xenial (16.04)
    x86_64/i386 deb package
    macOS (10.12 - 10.14) x86_64 -

    Note: the server-side Acra components are not compatible with Windows OS as host OS, consider using Docker.

    Compatibility and integration

    AcraServer is a server-side service that works as database proxy: it sits transparently between your application and the database, listens silently to all the traffic that's coming to and from the database. AcraTranslator is database-agnostic: it provides HTTP and gRPC API to decrypt AcraStructs stored anywhere.

    Acra is compatible with numerous RDBMS, object and KV stores, cloud platforms, external key management systems (KMS), load balancing systems.

    Cloud platforms DigitalOcean, AWS, GCP, Heroku
    RDBMS MySQL v5.7+, PosgtreSQL v9.4-v11, MariaDB v10.3
    Google Cloud SQL, Amazon RDS
    Object stores filesystems, KV databases, Amazon S3, Google Cloud DataStore
    Load balancing HAProxy, cloud balancers

    Open source Acra has limited integration support, more services are available in the Enterprise version of Acra only.

    Installation and launch

    Quick try (run example apps)

    Acra Example Projects illustrate the integration of Acra data protection suite into existing applications: web applications based on Django and Ruby on Rails frameworks, and simple CLI applications. We took well-known apps, detected sensitive data there and added the encryption layer. Protecting the data is completely transparent for the users and requires minimal changes in the infrastructure and application code.

    Developers and Ops friendly:

    • run a single command to deploy the application, database, Acra's components, logs, and dashboards;
    • read the code changes and see how little it takes to integrate encryption into the client application;
    • learn how Acra works by reading logs, monitoring metrics in Prometheus, checking tracers in Jaeger and watching Grafana dashboards;
    • inspect Docker-compose files, architecture schemes, database tables, and much more.

    Requirements: Linux or macOS with installed Docker.

    ⚙️ Run Acra Example Projects ⚙️

    Quick integration into your infrastructure

    For a quick and easy integration of Acra into your own infrastructure, we recommend trying Acra with Docker first. Using only two commands, you will get all the Acra's components and database up and running, with a secure transport layer between them. We prepared several typical infrastructure variants to experiment with.

    • Select one appropriate use case from the pre-made configurations ("Compose files"): use AcraServer-based configuration to protect the data in a database or select AcraTranslator to protect the files or any other binary blob stored elsewhere.
    • Launch Acra's server-side by running the selected Docker-compose file: it will generate the appropriate keys, put them into correct folders, perform a public key exchange, run selected services and database, and then it will listen to the incoming connections.
    • Integrate AcraWriter into your application code where you need to protect the sensitive data, supply AcraWriter with an Acra storage public key (generated by Docker-compose on the previous step). Encrypt the data into AcraStructs and send them into the database or file storage.
    • Decrypt data by reading the database through AcraServer or by decrypting the files through AcraTranslator.

    Please use the Acra Docker demo stand for testing/experimenting purposes only as the encryption keys are pre-generated in the configuration.

    Normal integration into your infrastructure

    For production environments, we insist on generating and exchanging keys manually and deploying Acra as Docker containers or from source code. Refer to the Quick Start guide to understand how to download and launch Acra components, generate keys, and perform the key exchange properly.

    Documentation, tutorials, additional information

    The most recent versions of the documentation, tutorials, and demos for Acra are available on the official Cossack Labs Documentation Server. The Github Wiki documentation is deprecated and is no longer updated since v0.82.0.

    To gain an initial understanding of Acra, you might want to:

    You can also check out the speaker slides for the following talks by Cossack Labs engineers:

    Demo projects

    Client-side encryption mode Transparent encryption mode

    AcraCensor: SQL firewall to prevent SQL injections

    AcraCensor example application illustrates how to use AcraCensor as SQL firewall to prevent SQL injections. The target application is a well-known vulnerable web application OWASP Mutillidae 2.

    The demo project has a Docker-compose file that runs the following web infrastructure:

    • OWASP Mutillidae web application,
    • Acra encryption suite.

    Acra works as a proxy between web and database. AcraCensor inspects every SQL query that runs from the web application to the database, and back, and blocks suspicious queries.

    Watch the video

    Requirements: Linux or macOS with installed Docker.

    ⚙️ Run AcraCensor SQL firewall example ⚙️

    Poison records: intrusion detection example

    Intrusion detection example application illustrates how to use poison records (honey tokens) as "intrusion markers" that warn about possible attack.

    Requirements: Linux or macOS with installed Docker.

    🍯 Run intrusion detection example 🐝

    Acra Load Balancing Example

    Acra load balancing example application illustrates building high availability and balanced infrastructure, based on Acra components, PostgreSQL, and Python web application. We prepared several configurations with multiple databases and HAProxy.

    🔛 Run Acra Load Balancing Example 🔛

    GDPR, HIPAA, CCPA

    Acra can help you comply with the current privacy regulations, such as:

    Configuring and using Acra in a designated form will cover most of the demands described in articles 25, 32, 33, and 34 of GDPR and the PII data protection demands of HIPAA. Read more about Acra and GDPR compliance here.

    Open source vs Pro vs Enterprise

    This is Acra Community Edition, the open source version of Acra, which is 💯 free for commercial and non-commercial usage. Please let us know in the Issues if you stumble upon a bug, see a possible enhancement, or have a comment on security design.

    There are also Pro and Enterprise versions of Acra available. Those versions provide better performance, redundancy/load balancing, come pre-configured with crypto-primitives of your choice (FIPS, GOST), have integration with key/secret management tools in your stack, provide unique security features like search through encrypted data, and have plenty of utils and tools for your Ops and SREs to operate Acra conveniently – deployment automation, scaling, monitoring, and logging. Talk to us to get full feature lists and a quote.

    Security consulting

    It takes more than just getting cryptographic code to compile to secure the sensitive data. Acra won't make you “compliant out of the box” with all the modern security regulations, and no other tool will.

    We help companies plan their data security strategy by auditing, assessing data flow, and classifying the data, enumerating the risks. We do the hardest, least-attended part of reaching the compliance – turning it from the “cost of doing business” into the “security framework that prevents risks”.

    Contributing to us

    If you’d like to contribute your code or provide any other kind of input to Acra, you’re very welcome. Your starting point for contributing is here.

    License

    Acra is licensed as Apache 2 open-source software.

    Contacts

    If you want to ask a technical question, feel free to raise an Issue or write to [email protected].

    To talk to the business wing of Cossack Labs Limited, drop us an email to [email protected].

    Blog Twitter CossackLabs Medium CossackLabs

    Issues
    • Add new linters to Acra CircleCI configuration

      Add new linters to Acra CircleCI configuration

      • Fix golint not being installed in prepare.sh and thus not reporting issues
      • Add misspell linter to detect common misspells
      • Add ineffassign linter to detect ineffectual assignments (assigned but not used or unintentionally assigned to wrong var because of how scopes and := work in Go)
      • Make all these linters write output to console, so we'll see what the issues are by looking at CI log

      I'm still unsure whether it's good idea to allow 10 ineffassign issues. There are currently 3, and those are intentional = nil assignments. Like, we won't see them until we reach 10, but then we won't be able to merge PR because CircleCI tells us there're too many issues

      docker infrastructure 
      opened by iamnotacake 11
    • TLS authentication with certificates as identifiers

      TLS authentication with certificates as identifiers

      Extended current TLS authentication where acra may starts with specified from cli params client_id which used for all incoming connections. This PR extends it and allow to start acra without static client_id and use data from certificates to map them to keystore identifiers/keys. PR contains 2 implementations of extracting data from certificates: common name and serial numbers, which should be chosen at startup (cli params not added yet). And one implementation of converting these values to acceptable format of ids using hex encoder (to allow non ascii printable values like serial numbers) and hash function (to allow values which longer than 256 length). A little bit refactored unit-tests to re-use same common function which verify wrapper's logic, added helpers to generate tls certificates in golang to test all specific cases related to certificate fields (@iamnotacake you can re-use them if need^)

      Done:

      • validation incoming certificate (deny CA certificates or with incorrect keyUsage/extKeyUsage values)
      • identifier fetching from certificates (distinguished name, serial numbers)
      • encoding identifier values to acceptable formats of clientID by sha512 and hex encoding

      Now AcraServer can use certificates from acra-connector connections when used tls_client_id_from_cert + acraconnector_tls_transport_enable, or from app's connection when used tls_client_id_from_cert + acraconnector_transport_encryption_disable. In both cases should be configured all necessary TLS params with ca/cert/key values. Plus user can configure which strategy of mapping certificate metadata to clientID to use. To configure should set tls_identifier_extractor_type with distinguished_name or serial_number value. In first case will be used "Subject" field ordered and formatted according to RFC2253, hashed with sha512 (as hash function which provide longest digest and least collision probability) and encoded to hex value. In second cases will be used serial number as 20-bytes integer value in big-endian order hashed with same function sha512 and encoded to hex value. First strategy good to use when certificates may be rotated and allow to avoid rotation keys and accordingly data too. Second strategy good for environments when Subject field not used in certificates.

      opened by Lagovas 9
    • AcraTranslator refactoring: service start up and shutdown, signals listener

      AcraTranslator refactoring: service start up and shutdown, signals listener

      This PR introduces several updates for our AcraTranslator service.

      In general I refactored main component (ReaderServer) and added support of SIGHUP handling to the service. What is refactored:

      • Added new StartFromFileDescriptor function that allows to run HTTP and gRPC servers having listener obtained from correspondent descriptor, instead of creating a new ones in order to preserve current listeners in new process;
      • Splitted monster Start function on lower components startHTTP, startGRPC in order to improve readability;
      • Wrapped Start and StartFromFileDescriptor with timeout exit (single point) and avoiding hidden os.Exit calls. This is extremely important for EE version that relies on CE ReaderServer component;
      • Moved from 'multiply listeners - multiply connections' to 'single listener - multiply connection' with transparent management of stopping/closing it. This is also important for better controlling of code execution;
      • Added some getters which is needed by EE version.

      There are two questions about previous implementation that were not obvious for me (most likely they are addressed to @Lagovas), and probably we can discuss it while review process: 1) why previously we had 'multiply listeners - multiply connection' model (when we called HandleClientConnection, it was a AcceptConnection function inside that created internally new listener and returned accepted connection via channel, their closing was managed via child contexts cancelling in separate goroutine) if listener itself allows calling Accept function safely from multiply goroutines; and 2) why does switching to new process works, if file descriptors created while SIGHUP processing and on service starting are different? (I asked @Lagovas about it and we came to the point that we know order number of Acra descriptor - it's 3, but it seems that we need not the order number but correct actual value of fd to start)

      How did I test this? Manually. I created a following methodology for testing it (locally run AcraTranslator, AcraConnector and 2 toy applications: one for sending HTTP and second - for gRPC requests):

      1. Test HTTP component without/with active connections;
      2. Test gRPC component without/with active connections;
      3. Test HTTP + gRPC components that run simultaneously without/with active connections;

      For sending HTTP requests I used the following piece of code (thanks to our documentation - but some examples are outdated there):

      for run in {1..100}; 
      do curl -X POST --data-binary @client.acrastruct --header "Content-Type: application/octet-stream" http://127.0.0.1:8000/v1/decrypt; 
      done
      

      For sending gRPC requests I used following go code:

      conn, err := grpc.Dial("127.0.0.1:8001", grpc.WithInsecure())
      	if err != nil {
      		t.Fatal(err)
      	}
      	defer conn.Close()
      
      
      	client := grpc1.NewReaderClient(conn)
      
      	for start := time.Now(); time.Since(start) < time.Second * 10; {
      		_, _ = client.Decrypt(context.Background(),
      			&grpc1.DecryptRequest{
      				ClientId:   []byte("testclientid"),
      				ZoneId:     []byte("test_zone"),
      				Acrastruct: []byte("blabla"),
      			},
      			grpc.EmptyCallOption{})
      	}
      

      While those applications were running, I sent (multiply times) SIGHUP signal to the AcraTranslator and observed that service restarted and there were no cut connections. Testing without connections are rather straightforward.

      acra-translator enhancement refactoring 
      opened by storojs72 8
    • Error with a golang dependency when using Go 1.10

      Error with a golang dependency when using Go 1.10

      Hello, I'm trying to build Acra from sources but I'm failing at the first step of the guide which is to install acra-keymaker.

      This is the error I get when I run go get github.com/cossacklabs/acra/cmd/acra-keymaker

      [email protected]:~/work/src/go.opencensus.io$ go get github.com/cossacklabs/acra/cmd/acra-keymaker
      package go.opencensus.io/exporter/jaeger: cannot find package "go.opencensus.io/exporter/jaeger" in any of:
              /usr/lib/go-1.10/src/go.opencensus.io/exporter/jaeger (from $GOROOT)
              /home/ubuntu/work/src/go.opencensus.io/exporter/jaeger (from $GOPATH)
      
      

      This got resolved when I downgraded go.opencensus.io/ from latest to v0.19.x because the package jaeger is moved to contrib.go.opencensus.io

      docs question 
      opened by Anirudh-MV 7
    • acra_rollback docs into wiki

      acra_rollback docs into wiki

      describe acra_rollback script

      acra-rollback 
      opened by gene-eu 6
    • Separate TLS config for AcraConnector and DB

      Separate TLS config for AcraConnector and DB

      Currently AcraServer uses only one TLS configuration for everything. This means, in particular, that both AcraConnector and DB are expected to use the same private CA, if root certificates are not available in the system certificate store. This is not always the case, and it would be nice to allow separate TLS configurations.

      This PR adds several new options:

      | New option | Old option | Purpose | | -- | -- | -- | | tls_client_ca | tls_ca | path to CAfile with trusted root certificates for AcraConnector verification | | tls_client_cert | tls_cert | path to server certificate presented to AcraConnectors | | tls_client_key | tls_key | path to private key to the certificate above | | tls_client_auth | tls_auth | TLS authentication level when accepting AcraConnector connections |

      | New option | Old option | Purpose | | -- | -- | -- | | tls_database_ca | tls_ca | path to CAfile with trusted root certificates for database verification | | tls_database_cert | tls_cert | path to client certificate used when connecting to database | | tls_database_key | tls_key | path to private key to the certificate above | | tls_database_auth | tls_auth | TLS authentication level when connecting to database | | tls_database_sni | tls_db_sni | server name expected when connecting to database (renamed option) |

      New options override old options. The old options still remains not deprecated, they might be useful as a shortcut.

      This required a refactoring to split the TLS configuration, ~~so in the future it will be possible to have more specific configuration (think, different allowed authentication modes, cryptosuites, TLS versions, etc.)~~ which allowed to easily split the configuration.

      There are no integration tests for this option currently. (Do we need them?) I have tested it manually, with recently updated certificates. The timing could not have been better...

      acra-server enhancement 
      opened by ilammy 6
    • Improve Acra Go Report scores

      Improve Acra Go Report scores

      • Fix gofmt issues (apply suggestions by gofmt)
      • Fix misspells
      • Fix golint issues (add comments to exported types and functions)
      • Fix ineffassign issues (those are assignments to variables that are unused or rewritten afterwards)
      • Fix two bugs (thanks to ineffassign checks):
        • In cmd/acra-server/client_commands_session.go the function was returning instead of breaking switch and reporting error to client
        • In decryptor/mysql/response_proxy.go we had := that overrides the scope so the new err was created instead of writing to already defined one
      • Zeroize sensitive data: add explicit calls to FillSlice to rewrite sensitive buffers with zeros before assigning them to nil
      • Handle possible errors of keys.New() in tests

      It turns out that CircleCI scripts are a little bit broken and golint check was not performed automatically when PRs were created/updated. Gonna fix that in different PR as well as add misspell checker, and maybe ineffassign one.

      bug refactoring 
      opened by iamnotacake 6
    • Generate keys with

      Generate keys with "acra-keys generate"

      Add a new subcommand for key generation. This is a replacement for acra-keymaker and other tools like acra-addzone.

      Generating master keys

      The first step is to generate a master key:

      acra-keys generate --master_key_path=master.key --keystore=v2
      

      The --keystore option is required since the format of master keys is different for keystore versions.

      The resulting master key is placed into the specified file. It needs to be set in the environment:

      ACRA_MASTER_KEY=$(cat master.key | base64)
      

      Generating initial keystore

      After you have a master key, generate the initial keystore:

      acra-keys generate --keystore=v2 --client_id=ACME
      

      The --keystore option is required here as well since we need to know which format to use. When the keystore is already initialized, this option is not necessary as acra-keys can see the format for itself.

      The --client_id option is now required as well. This is different from acra-keymaker which uses client ID client when it is not specified explicitly.

      acra-keys will generate a default set of keys:

      • transport keys for AcraConnector, AcraServer, AcraTranslator
      • storage key for AcraWriter to use

      It is also possible to initialize the keystore with only some of the keys. For example, if you need only the storage key without the transport keys, initialize the keystore like this:

      acra-keys generate --keystore=v2 --client_id=ACME --client_storage_key
      

      Note that key options are different from the ones used by acra-keymaker.

      | Option | Key | | -- | -- | | --acraconnector_transport_key | transport keypair for AcraConnector | | --acraserver_transport_key | transport keypair for AcraServer | | --acratranslator_transport_key | transport keypair for AcraTranslator | | --client_storage_key | encryption keypair for AcraWriter | | --acrawebconfig_symmetric_key | encryption key for AcraWebConfig |

      The keystore will be initialized in the default directory: .acrakeys in the current directory. You can change this with --keys_dir (and --keys_dir_public for v1). Note that these option names are different from acra-keymaker which uses --keys_output_dir. This is to be consistent with other acra-keys subcommands.

      Rotating existing keys

      acra-keys generate can be used to rotate keys in the existing keystore. Just run it again and specify the keys you need to rotate:

      acra-keys generate --client_id=ACME --acraserver_transport_key
      

      This will generate a new key for the ACME client to use on AcraConnector when connecting to AcraServer.

      New features for zones

      acra-keys generate can also be used instead of acra-addzone to generate new zones:

      $ acra-keys generate --zone
      {"id":"DDDDDDDDtapktGNSfSnodqtv","public_key":"VUVDMgAAAC1UemR1A1qdHwVPp4iFTZKnHFNOMQqHmUc04SFGxwUk3wnmY0ut"}
      INFO[0000] Generated new Acra zone
      

      JSON output format is the same as acra-addzone.

      You can also use it to rotate existing zone keys (previously this was impossible to do):

      $ acra-keys generate --zone_id=DDDDDDDDtapktGNSfSnodqtv --zone_storage_key
      {"id":"DDDDDDDDtapktGNSfSnodqtv","public_key":"VUVDMgAAAC0cIjpMAoXUNFamoZqCDaY9wAHAT/tXhfWOqTAYDqPp5PZAXtwB"}
      INFO[0000] Generated zone storage key               
      

      Compatibility with acra-keymaker and its replacement

      Note that the command-line is different and not compatible with acra-keymaker. The plan is to eventually deprecate acra-keymaker in favor of acra-keys. We could possible reimplement acra-keymaker and acra-addzone using the shared code from acra-tools to reduce maintenance burden. For now, both acra-keymaker and acra-keys generate will be supported, with the latter being the preferred way to generate keys.

      Pending tasks

      • [x] Review by @Lagovas
      • [x] Review by @shad
      • [x] Review by @vixentael
      • [x] Adjust this PR for Acra EE
      • [x] Rename --new_zone => --zone
      enhancement key-management 
      opened by ilammy 6
    • Fix Go 1.15 support with new test certificates

      Fix Go 1.15 support with new test certificates

      This PR fixes a recently found issue with TLS tests failing after updating to Go 1.15.

      Backstory

      We have decided to update Go versions used for testing and accidentally discovered a regression with Go 1.15. Integration tests with TLS started to fail with exceptions like

        File "/home/user/.local/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 481, in connect
          return self.dbapi.connect(*cargs, **cparams)
        File "/home/user/.local/lib/python3.5/site-packages/psycopg2/__init__.py", line 126, in connect
          conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
      sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) SSL SYSCALL error: EOF detected
      
      (Background on this error at: http://sqlalche.me/e/e3q8)
      

      The issue did not reproduce with Go 1.14 and it was weird.

      Our initial assessment of the changelog has pointed some updates in crypto/tls which – spoiler alert! – turned out to be a red herring.

      What actually went wrong

      Well, I've reproduced the issue in CircleCI environment but that did not help very much. So I just started AcraServer built with Go 1.15, configured AcraConnector, a database with TLS, etc., and... here's what I saw in AcraServer's logs when trying to access the database via AcraConnector:

      CEF:0|cossacklabs|acra-server-ee|0.85.0|584|Can't initialize tls connection with db|6|client_id=keypair1 decrypt_mode=wholecell error=x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG\=x509ignoreCN\=0 proxy=server unixTime=1600779074.282 
      CEF:0|cossacklabs|acra-server-ee|0.85.0|584|Can't process SSL request|6|client_id=keypair1 decrypt_mode=wholecell error=x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG\=x509ignoreCN\=0 proxy=server unixTime=1600779074.283 
      

      Quick testing has shown that setting GODEBUG=x509ignoreCN=0 makes the issue go away, even for integration tests. Another quick look has shown that the certificates we use indeed do not contain Subject Alternate Names (SAN).

      Another look at the patch notes revealed that Go 1.15 drops legacy support for CommonName as host name if there are no Subject Alternate Names in the certificates.

      How to fix this issue

      Instead of putting localhost into Common Name, add a Subject Alternate Name with that host name, and use something which does not look like a hostname in Common Name.

      Here is how the old certificates look:

      Old CA certificate
      $ openssl x509 -text -noout -in tests/ssl/ca/ca.crt
      Certificate:
          Data:
              Version: 3 (0x2)
              Serial Number:
                  3c:4b:87:bf:7b:a7:ef:16:25:7f:70:84:a1:e6:bd:d2:8d:8e:f8:73
              Signature Algorithm: sha256WithRSAEncryption
              Issuer: C = GB, ST = London, L = London, O = Global Security, OU = IT, CN = ca-localhost
              Validity
                  Not Before: Jan 31 12:48:33 2019 GMT
                  Not After : Jan 18 12:48:33 2069 GMT
              Subject: C = GB, ST = London, L = London, O = Global Security, OU = IT, CN = ca-localhost
              Subject Public Key Info:
                  Public Key Algorithm: rsaEncryption
                      RSA Public-Key: (2048 bit)
                      Modulus:
                          00:ee:97:df:c7:c4:52:e2:79:55:41:c9:b4:8c:e7:
                          bc:34:9f:7f:05:74:67:b6:ea:46:45:17:ba:36:6e:
                          e2:63:b6:65:5b:be:44:60:31:7a:7c:f1:02:96:a0:
                          c9:52:11:1d:c2:36:3b:95:06:8f:18:a4:f0:94:a3:
                          c2:b8:4f:63:c6:f8:f3:c6:2f:c5:7f:ee:dc:0c:24:
                          d7:15:1c:f7:4e:ca:b0:79:96:b6:25:c3:0b:76:1d:
                          d1:ba:9d:a7:df:61:0b:4b:48:f0:4b:b8:4e:45:75:
                          f3:53:d5:f9:23:86:73:9f:b6:a6:6e:22:d6:83:e5:
                          3d:07:b6:8e:9e:f0:77:ef:13:60:e2:4b:6e:c3:2e:
                          6f:15:d9:6c:b6:01:e7:a7:99:e7:f6:2c:a4:e5:68:
                          a2:94:1f:5f:be:18:0d:cb:9c:a8:bb:66:71:e8:45:
                          14:e0:14:bc:87:21:f5:e4:f1:5a:5c:0d:4e:1d:25:
                          3a:02:c5:ff:25:df:b9:21:e6:f9:4b:70:5d:22:ee:
                          81:54:e0:9a:56:19:ae:9e:fd:45:7b:bd:1c:da:a8:
                          fc:97:c8:a5:53:0a:07:f4:74:98:a8:02:ec:ab:5d:
                          5f:f4:ec:5e:5b:1a:08:57:45:de:96:01:63:6d:61:
                          f5:fd:c4:bb:1f:d5:8a:5a:9f:29:8e:2e:4c:fe:e9:
                          fe:ff
                      Exponent: 65537 (0x10001)
              X509v3 extensions:
                  X509v3 Subject Key Identifier: 
                      21:74:60:D6:C2:B2:2C:B4:06:D8:05:10:C2:03:7A:A8:35:BC:36:FC
                  X509v3 Authority Key Identifier: 
                      keyid:21:74:60:D6:C2:B2:2C:B4:06:D8:05:10:C2:03:7A:A8:35:BC:36:FC
      
                  X509v3 Basic Constraints: critical
                      CA:TRUE
          Signature Algorithm: sha256WithRSAEncryption
               d2:ae:2a:58:b9:18:05:19:07:94:b9:1c:16:1c:d3:cc:99:27:
               80:c0:27:7b:f6:12:3d:cb:0d:0c:fb:da:30:8c:0d:82:c0:d8:
               b2:1f:b2:0e:74:fb:86:08:32:98:32:ac:ed:09:bc:ba:83:9c:
               75:a4:81:83:a5:34:76:4d:80:4c:e8:39:05:47:87:ed:76:a2:
               53:fa:9d:ce:11:91:54:20:50:6e:da:90:35:5f:79:0a:0c:6a:
               bd:c3:43:6e:86:8c:85:d0:09:31:39:7e:66:e6:89:fd:7f:12:
               b7:60:a2:41:20:06:96:2e:b2:e9:8d:ef:e1:55:bf:ba:61:80:
               27:a0:62:f2:3a:16:78:21:89:a0:cf:24:a3:e4:a6:5d:d8:5a:
               2b:9a:31:bc:3a:ed:5c:07:85:0c:8b:c0:88:b9:d6:ee:43:49:
               bb:86:8e:9e:32:c0:c3:ed:80:09:68:e6:e6:35:8e:10:e9:68:
               da:69:bc:c0:a6:8d:e2:ab:fb:26:5e:91:f0:d9:f5:4f:82:24:
               d9:c3:df:51:61:d6:40:3e:fb:c9:4f:5d:d0:8e:07:22:78:93:
               a9:f6:ac:aa:5d:18:f8:e7:fb:88:2d:0d:47:42:9e:dd:87:97:
               dd:71:c8:5c:5c:26:bd:cc:f7:51:6d:3f:94:88:c7:92:c0:6f:
               59:8a:a8:25
      
      Old client certificate
      $ openssl x509 -text -noout -in tests/ssl/acra-writer/acra-writer.crt 
      Certificate:
          Data:
              Version: 1 (0x0)
              Serial Number:
                  04:90:aa:e8:b4:38:54:1e:16:3b:7a:9a:c1:70:fb:83:78:f5:cd:0e
              Signature Algorithm: sha256WithRSAEncryption
              Issuer: C = GB, ST = London, L = London, O = Global Security, OU = IT, CN = ca-localhost
              Validity
                  Not Before: Jan 31 12:48:34 2019 GMT
                  Not After : Jan 18 12:48:34 2069 GMT
              Subject: C = GB, ST = London, L = London, O = Global Security, OU = IT, CN = localhost
              Subject Public Key Info:
                  Public Key Algorithm: rsaEncryption
                      RSA Public-Key: (2048 bit)
                      Modulus:
                          00:db:c7:c4:cc:eb:fb:e4:27:9f:ca:63:0f:82:1c:
                          26:5b:f2:43:35:e3:5a:be:3f:a4:56:5e:91:48:03:
                          ce:0b:7e:cf:c2:de:ba:f2:ca:25:42:42:71:ca:7b:
                          49:7d:91:04:ff:a3:49:2c:bb:28:aa:61:b5:62:ed:
                          e2:7a:5f:d8:ae:e2:7a:68:66:2c:74:99:95:49:5d:
                          6c:b2:ba:81:0f:0a:ea:a3:e9:97:14:ae:41:98:14:
                          da:19:d3:af:c0:5d:59:dc:9d:9f:2c:7b:52:3e:ef:
                          95:ea:c5:5d:1d:60:54:40:2c:a9:fb:40:5b:17:e2:
                          2e:af:88:75:88:a4:be:65:cc:72:15:2b:0f:b8:ab:
                          81:78:f9:52:ec:9a:d4:9a:ac:12:2c:3a:b3:cd:5d:
                          8f:bc:55:73:63:71:01:f0:f2:b7:f5:83:e3:01:90:
                          c0:d7:10:6a:39:47:89:86:9c:09:79:ab:db:bf:04:
                          d9:eb:ed:da:6f:54:70:86:9e:9b:e2:a0:df:8a:a1:
                          fb:21:7c:3e:bb:52:f6:77:2e:b4:55:21:e3:52:1d:
                          ce:26:7e:2d:32:58:07:6b:71:0b:59:ca:08:30:66:
                          e0:f9:97:79:e6:81:13:07:ed:b2:ff:eb:08:a6:0a:
                          fb:a7:0b:84:09:cf:6e:48:ec:0f:e1:76:72:23:46:
                          6a:33
                      Exponent: 65537 (0x10001)
          Signature Algorithm: sha256WithRSAEncryption
               11:97:17:70:34:b0:93:99:f5:c7:2d:f4:8a:61:15:8c:db:b6:
               9a:45:22:f4:e4:a6:09:7d:45:10:99:52:be:a9:f7:9b:94:e6:
               f5:5a:97:63:61:4d:76:48:d6:61:87:e8:dc:9b:b4:e1:ab:68:
               b2:5d:ac:5c:7d:70:40:9b:07:7a:f8:0f:21:5f:7a:9d:94:b2:
               ae:6d:a9:dd:c6:a1:5c:d2:17:c2:70:38:a4:9b:62:a4:66:73:
               3b:ed:46:c7:8b:a4:bd:2c:b9:f8:02:81:4f:f7:34:8c:6a:41:
               31:7d:2e:5a:36:79:17:75:d7:a2:e4:53:a3:58:81:6a:82:e9:
               17:44:d5:e2:85:3c:59:b9:77:21:7d:2c:96:74:7f:2b:de:e9:
               f4:30:a0:d0:59:49:0c:32:47:8b:fe:34:29:5f:fa:0c:bc:67:
               32:89:4e:7d:77:e8:67:e1:af:64:b9:2b:c5:ed:74:5e:99:2f:
               e1:37:b9:75:c4:d2:30:52:d9:16:af:3f:29:6b:b4:12:a0:a9:
               97:03:75:9f:d1:a1:f9:0f:e3:07:5d:8a:75:dd:52:19:6a:ce:
               29:2f:14:fb:30:51:46:2e:8d:10:22:f3:d1:c7:9f:96:18:66:
               58:a4:11:de:c8:31:f1:17:e5:47:d6:a7:78:5c:6b:bf:fb:0b:
               cf:9b:be:ab
      

      And here are the new ones:

      New CA certificate
      $ openssl x509 -text -noout -in tests/ssl/ca/ca.crt 
      Certificate:
          Data:
              Version: 3 (0x2)
              Serial Number:
                  43:6e:0c:5b:e4:aa:df:a0:1f:01:4c:ba:46:ad:61:45:81:c8:ec:28
              Signature Algorithm: sha256WithRSAEncryption
              Issuer: C = GB, ST = London, L = London, O = Global Security, OU = IT, CN = Test CA certificate
              Validity
                  Not Before: Sep 22 16:05:34 2020 GMT
                  Not After : Sep 10 16:05:34 2070 GMT
              Subject: C = GB, ST = London, L = London, O = Global Security, OU = IT, CN = Test CA certificate
              Subject Public Key Info:
                  Public Key Algorithm: rsaEncryption
                      RSA Public-Key: (2048 bit)
                      Modulus:
                          00:be:12:60:a8:38:dc:80:be:30:2f:5b:0a:b1:55:
                          36:94:0a:43:c3:03:3a:36:7f:ae:f1:83:e8:69:a4:
                          e2:73:26:e0:4e:5d:e7:79:cc:1b:df:4a:d1:88:4c:
                          85:af:0b:34:ad:ad:22:ba:3d:22:c4:e8:9e:99:33:
                          51:4d:8e:99:49:68:3f:ee:44:e4:51:58:0f:37:48:
                          0a:aa:3a:cc:e3:c8:3c:fe:41:db:3b:a7:fc:ec:a8:
                          4b:bd:99:a0:56:3b:73:0a:d0:52:e5:f5:c1:0d:87:
                          87:07:f9:78:4c:fe:ab:3c:09:7b:06:a4:3c:fa:1c:
                          c0:d4:83:27:5b:44:27:64:76:55:01:b3:e9:30:2b:
                          86:a6:8e:12:9e:de:65:9b:25:22:9d:89:da:98:03:
                          f3:47:cd:84:ee:58:84:98:c9:d2:a2:e3:31:a9:47:
                          1b:a8:dc:08:eb:73:f3:ae:ce:d4:78:66:ba:19:8e:
                          91:4e:40:82:e7:63:6a:bd:17:05:56:45:1f:b7:96:
                          c6:7c:10:95:46:07:64:d3:08:c9:ce:56:70:74:dc:
                          77:46:56:c4:29:c6:0a:27:dd:f3:1e:43:27:7a:bc:
                          36:87:9c:8c:38:f1:75:21:04:f6:95:a1:ab:2b:ee:
                          25:48:9d:1f:71:91:9a:bf:1e:5a:17:48:7a:99:86:
                          98:0b
                      Exponent: 65537 (0x10001)
              X509v3 extensions:
                  X509v3 Basic Constraints: critical
                      CA:TRUE
                  X509v3 Subject Alternative Name: 
                      DNS:localhost
                  X509v3 Subject Key Identifier: 
                      E9:58:50:C8:E6:25:61:E2:D6:B2:51:5B:21:BD:49:BB:83:A0:4A:A8
                  X509v3 Authority Key Identifier: 
                      keyid:E9:58:50:C8:E6:25:61:E2:D6:B2:51:5B:21:BD:49:BB:83:A0:4A:A8
      
          Signature Algorithm: sha256WithRSAEncryption
               06:a8:86:5b:a3:c0:e0:c1:04:08:59:8a:bb:cd:98:ac:04:2b:
               54:dd:b4:eb:de:a9:9c:86:c9:3d:32:cd:a0:b7:5a:82:3e:2d:
               5c:17:11:5c:ef:3b:83:d7:a7:7e:e2:5f:5c:02:f6:e8:52:47:
               bd:f9:f0:cc:32:24:1e:42:22:d7:4a:01:25:92:e3:43:fe:81:
               43:04:7e:8c:34:84:5d:e4:5f:0d:a8:c0:e8:b8:4e:e6:20:d1:
               a7:d0:bf:0e:f7:f3:72:7a:56:eb:22:0e:13:eb:a0:76:ba:e8:
               08:71:a4:e6:4e:5a:6e:3c:06:21:2b:a8:6c:98:13:73:30:df:
               fa:a5:a9:00:61:29:d4:9a:15:3e:de:10:68:22:a7:17:03:5b:
               5c:96:b7:ed:77:f2:41:a8:cf:ec:45:59:95:41:ca:af:81:56:
               71:a3:5d:d9:98:73:4d:53:2d:42:53:33:bf:90:ea:73:b0:81:
               4f:b6:b5:f9:a2:ed:b8:71:17:d4:17:59:44:17:4d:7f:d3:b0:
               bf:b0:ef:6e:22:4e:dd:d3:2a:db:db:6b:f9:11:b2:8f:c8:c5:
               2e:f8:bb:09:45:bd:bf:0f:d2:d6:b6:0e:b0:6f:ad:81:d4:1c:
               e9:6a:73:8f:79:2e:a5:17:5f:c0:35:93:db:93:58:33:25:9d:
               57:9c:10:e8
      
      New client certificate
      $ openssl x509 -text -noout -in tests/ssl/acra-writer/acra-writer.crt
      Certificate:
          Data:
              Version: 3 (0x2)
              Serial Number:
                  3b:84:c1:a2:8a:44:d9:5f:99:ac:0b:d1:ef:19:af:36:6b:cb:b3:51
              Signature Algorithm: sha256WithRSAEncryption
              Issuer: C = GB, ST = London, L = London, O = Global Security, OU = IT, CN = Test CA certificate
              Validity
                  Not Before: Sep 22 16:05:34 2020 GMT
                  Not After : Sep 10 16:05:34 2070 GMT
              Subject: C = GB, ST = London, L = London, O = Global Security, OU = IT, CN = Test leaf certificate
              Subject Public Key Info:
                  Public Key Algorithm: rsaEncryption
                      RSA Public-Key: (2048 bit)
                      Modulus:
                          00:de:fa:c7:73:11:6f:6d:3f:67:13:dc:b7:b7:71:
                          ae:28:7b:b3:0c:0e:3a:6a:97:9c:31:87:25:0b:fc:
                          6f:85:45:74:d9:4b:0f:a1:4e:85:b9:3e:94:e5:9f:
                          60:46:39:56:cc:aa:59:b3:f5:e6:51:80:c5:d5:a5:
                          6b:d3:77:f1:67:15:6b:3e:3f:db:41:0e:ba:62:ca:
                          d3:26:9a:50:05:de:9f:a8:45:6c:61:14:d6:87:33:
                          e1:d1:21:9e:61:4c:85:c7:c1:72:50:80:e2:42:70:
                          98:08:e9:19:22:40:bf:11:09:3b:61:ee:7b:52:3a:
                          60:8a:f5:0f:65:87:8d:ec:b4:ef:c4:05:64:25:4e:
                          8b:33:0a:86:6a:f7:41:f1:60:6b:0d:f1:15:98:90:
                          40:d7:10:21:db:3a:b5:cf:70:94:99:7c:bb:37:fa:
                          de:e2:f2:30:f2:82:f2:2a:fe:f0:ab:e5:56:f1:14:
                          99:92:5a:5c:f9:86:e1:cc:39:d2:b4:0f:bc:2d:47:
                          a6:2d:f4:77:52:fa:3e:43:49:96:fa:5a:56:e9:c9:
                          32:78:c8:d3:af:c6:7a:ed:cf:35:a6:26:c7:0f:1f:
                          d8:db:96:b8:5a:23:2a:cc:fe:93:ec:17:46:e2:11:
                          fe:12:81:ab:26:07:2d:fc:77:4d:8d:fb:9b:0c:76:
                          92:23
                      Exponent: 65537 (0x10001)
              X509v3 extensions:
                  X509v3 Basic Constraints: 
                      CA:FALSE
                  X509v3 Key Usage: 
                      Digital Signature, Non Repudiation, Key Encipherment
                  X509v3 Subject Alternative Name: 
                      DNS:localhost
          Signature Algorithm: sha256WithRSAEncryption
               a3:78:b0:f4:39:b4:b4:f9:67:69:70:25:55:60:5f:3b:93:97:
               22:67:82:c1:52:a7:56:29:05:d1:d4:fd:44:56:c5:79:7b:8a:
               b2:7b:5d:9f:1c:92:5f:a8:a0:cc:19:a6:4b:66:23:8d:a7:53:
               57:f6:7e:68:95:d0:78:8c:f6:1f:36:aa:ba:3d:74:c8:74:be:
               e4:28:39:0c:9c:27:da:bd:c0:a7:2c:7b:2d:1b:ab:a7:07:6a:
               e2:16:c7:c9:87:da:38:42:95:33:09:ea:f6:e5:a3:1f:00:de:
               40:bc:de:1a:b6:c6:ea:eb:31:e5:68:a8:e6:53:91:12:a7:0c:
               0a:05:72:06:be:8b:5e:f0:ca:fe:e3:54:ba:ba:cf:a0:43:f7:
               ab:02:3c:65:95:e4:c4:9a:b7:ad:d2:ad:0c:d5:14:0f:1e:24:
               d1:ff:e6:ee:c4:ac:ea:3f:a0:da:f1:fc:5f:59:43:88:8c:d3:
               84:d4:07:a6:3e:07:b6:5b:76:11:47:d2:82:b7:67:35:e1:d8:
               32:df:00:35:56:c5:5f:1b:ae:e0:c1:a0:5c:40:94:19:93:19:
               c6:24:11:df:23:38:cd:32:4a:cb:3d:27:29:0d:6f:d0:4d:4f:
               b4:79:aa:da:cd:c7:87:15:ad:99:30:12:a1:48:46:dd:2a:b5:
               26:bc:54:18
      

      The interface for setting these extension variables is not a paragon of usability, but OpenSSL is what it is. We need to provide a configuration file and pass the section names to use. Since those are test certificates, we can use only bare minimum of fields. It took a while to figure out the correct settings.

      Test certificates and private key files are updated and committed into the repository. This will trigger recently added GitHub alert about that. You can disregard it.

      Updated Docker images

      The database images we use contain the certificates and keys generated here, in Acra CE. Switch to updated versions of the images which contain the freshly generated data.

      | Database | Old tag | New tag | | -- | -- | -- | | PostgreSQL | cossacklabs/postgresql-ssl:11 | cossacklabs/postgresql-ssl:11-1 | | MySQL | cossacklabs/mysql-ssl:5.7.25 | cossacklabs/mysql-ssl:5.7.25-1 | | MariaDB | cossacklabs/mariadb-ssl:10.3 | cossacklabs/mariadb-ssl:10.3-1 |

      Since the images are tied to contents of tests/ssl, new tags are introduced and used. After all branches of CE and EE have migrated to the new certificate files it will be possible to update the base tags. (Or, most likely, we'll leave them as it).

      The images have been manually built and uploaded to Docker Hub by me. The Dockerfiles can be found in tests/docker.

      Updated Go versions

      Finally, update the Go versions that we use:

      • 1.15 – latest stable version
      • 1.14 – the previous one
      • 1.13 – the oldest version we support

      (Right now the oldest version is 1.11, but #414 bumps that to 1.13 so I'm using it to avoid conflicts.)

      acra-server infrastructure tests 
      opened by ilammy 5
    • Adding data type to encrypted column in AcraServer encryption configuration

      Adding data type to encrypted column in AcraServer encryption configuration

      issues:

      In the documentation, this is mentioned :

      Transparent encryption proxy mode allows you to configure AcraServer to encrypt records in specific database columns without altering the application code.
      

      However, There is not possibility for my not editable application to work with this proxy as :

      Encrypted data is binary data. As AcraServer doesn't know the nature of data, it returns the decrypted binary data to the web app. You’ll need to change the source code of your web app for the app to expect the decrypted data as binary, then to encode it into the original format (strings, numbers, bytes, etc.).
      

      Expected behaviour :

      As a transparent proxy user , I expect to have to define the database field type either in the acra server/connector configuration and in the database (change encrypted field into binary field) but not in the application code.

      Regards,

      acra-server enhancement feature-request 
      opened by wdesplas 2
    • AcraCloud: Admin/instance config UI

      AcraCloud: Admin/instance config UI

      Interface, that allows to configure (generate configuration, attach storage) and fire up a new instance.

      cloud distant future feature-request 
      opened by gene-eu 1
    • rollback decryption for injected acrastructs

      rollback decryption for injected acrastructs

      pieces off https://github.com/cossacklabs/acra/issues/51

      acra-rollback feature-request 
      opened by gene-eu 1
    • Poison record rotation

      Poison record rotation

      Think about this closer to 1.0, talk to users.

      If we do:

      When triggered poison record rotation:

      • acraserver generates new poison records
      • user feeds select that triggers poison records
      • acra reads output, replaces poison records with old keys with poison records with new keys, generates a dump of inserts.
      • user receives sql commands and is able to run it / server runs it for user?
      acra-poison_records_maker core distant future feature-request 
      opened by gene-eu 1
    • Default zone

      Default zone

      Working without zones is actually working with a default zone. We might want to unify the terminology at some point, and streamline the process on AcraServer side.

      core distant future feature-request 
      opened by gene-eu 2
    • Random zone pools

      Random zone pools

      Closer to ending existing line of problems, allow users to use Zones as source of Random keys.

      • generateZonePool of size N for application XYZ
      • getRandomZoneFromPool for application XYZ
      • getZoneFromPool for application XYZ where zone_id is “ABC”
      core distant future feature-request 
      opened by gene-eu 0
    • process authentication

      process authentication

      As we will need this is Hermes's ecosystem at some point, it's wise to study process authentication for platforms which enable it well.

      core feature-request 
      opened by gene-eu 0
    Releases(0.85.0)
    • 0.85.0(Mar 15, 2019)

      0.85.0, March 15th 2019

      Core:

      • Breaking changes:

        Introducing a new more flexible configuration format for AcraCensor rules. AcraCensor doesn't support the old format, all users should migrate (don't worry, it's a simple procedure).

      • Search through encrypted data

        You now can run SQL queries over encrypted AcraStructs allowing users to search through sensitive data without exposing it. This feature is only available in Acra Enterprise version.

      • Transparent proxy mode

        TLDR: Transparent proxy mode allows you to configure AcraServer to encrypt records in specific database columns without altering the application code.

        The application flow doesn't need to change: application sends SQL requests through AcraConnector and AcraServer to the database. AcraServer parses each request, encrypts the desired values into AcraStructs, and passes the modified requests to the database. To retrieve the decrypted data, your application talks to AcraServer again: upon receiving the database response, AcraServer tries to detect AcraStructs, decrypts them, and returns the decrypted data to the application.

        Transparent proxy mode is useful for large distributed applications where updating the source code of each client app separately would be complicated.

        To enable this mode, you need to create a separate encryptor configuration file (acra-encryptor.yaml) that describes which columns to encrypt and provide a path to it in the AcraServer configuration file (or via CLI params --encryptor_config_file=acra-encryptor.yaml).

        Read more details in the Readme and in the Acra documentation section dedicated to Transparent encryption.

        (#285, #309, #314).

      • AcraCensor – SQL firewall to prevent SQL injections

        TLDR: Improved stability of AcraCensor, switched to more flexible rules' configuration.

        Breaking changes: Introducing a new format for configuration files, the previous format is no longer supported, you should migrate to the new one.

        • New configuration file format allows configuring the allowlist and the denylist separately or simultaneously.

          The allow handler allows something specific and restricts/forbids everything else. The allowall handler should be a final statement as that means that all the other queries will be allowed.

          The deny handler allows everything and forbids something specific. The denyall means "block all queries!" (that haven't been allowed or ignored before).

          For each handler, there are settings that regulate queries, tables, and patterns. The order of priority for the lists is defined by their position in the configuration file. The processing priority for each list is as follows: queries, followed by tables, followed by patterns.

          (#298, #297, #304, #306).

          Read more in AcraCensor docs.

        • Added version to the configuration file. This allows detecting an outdated configuration easily. From now on, AcraCensor supports explicit configuration version and logs errors if the configuration is not valid (#321).

        • Improved parsing of SQL queries with prepared statements (#303, #283).

        • Improved error handling for queries that AcraCensor can't parse (#291, #284).

        • Added ability to log unparsed queries to a separate log file for the debugging and configuration purposes. Sometimes AcraCensor can't parse all of the incoming queries and it is useful to have a separate log for them.

          How to use it: Provide the path to the unparsed queries log file in the configuration file parse_errors_log: unparsed_queries.log (#295).

        • Improved support of PostgreSQL queries ("RETURNING" clause) and quoted identifiers (now you can use "tablename" and WHERE "column"=1) (#296).

        • Fixed the bug in QueryCapture log that caused duplicated of records in the log to appear (#318).

      • AcraServer

        • Fixed handling of null-size packets in PostgreSQL protocol (#286).

        • Fixed handling of setting a custom connection API port (#294).

        • Fixed handling of the plain text data response: if the database returns a plain text response, it is redirected "as is" (#305).

        • Fixed handling of casted placeholders in expressions like SELECT $1::type1::type2 FROM table1 WHERE column1=$2::type3::type4 (#328).

        • Improved code quality (some refactoring here and there) (#302, #301).

      • AcraServer, AcraTranslator, AcraConnector

        • Refactored logs and error messages got even more descriptive and user-friendly (#312, #299, #317).

        • Added on-start version logging to make it easier to understand which version is running (#319).

        • Added versioning for configuration files of each service (#322).

        • Added exporting version to metrics (#330, #320).

        • Updated some configuration parameters descriptions for better user-friendliness (please see our docs of AcraConnector and AcraServer for detailed descriptions of each parameter and usage examples) (#329).

      • AcraWriter

        • Updated AcraWriter for ActiveRecord (Ruby), fixed dependencies, added support of mysql2 adapter (#287).

        • Updated AcraWriter for Django (Python), fixed potential encoding issues (#293, #292).

        • Updated AcraWriter for C++, improved cpp codec usage (#290, #289).

        • Added bitcode for AcraWriter iOS and added Swift example project (#327, #326, #325, #324, #323, #323, #307).

        • Improved distribution of AcraWriter for Android, now it's available via Maven (#310).

      • Other

        • Added more tests and then — added even more tests. We just love automating things! (#331, #311, #308, #292).

        • Updated the version of pyyaml used in the tests due to CVE-2017-18342. This change doesn't affect the users of Acra, it only affects our test suite (#300).

      Infrastructure:

      • Updated Docker files, added more comments, and updated Go version (#313, #288).

      Example projects and demos:

      Related blog posts:

      Features coming soon:

      • Pseudonymisation: an early version of pseudonymisation library/plugin for Acra for transparent data pseudonymisation.

      • Cryptographically protected audit log: protection for logs against tampering.

      Documentation:

      Source code(tar.gz)
      Source code(zip)
    • 0.84.2(Feb 19, 2019)

      0.84.2, February 19th 2019

      Hotfix:

      Fixed an issue in communication of AcraServer and PostgreSQL that causes AcraServer to stop processing connection due to an unexpected error in parsing packets. The issue occurred when last data in data row column from PostgreSQL comes with empty data (0 bytes).

      Details: https://github.com/cossacklabs/acra/pull/315

      Source code(tar.gz)
      Source code(zip)
    • 0.84.1(Jan 24, 2019)

      0.84.1, January 25th 2019

      Hotfix:

      Fixed an issue in communication of AcraServer with some specific ORMs (xorm precisely) with MySQL database. In some cases, when database has plaintext data, AcraServer can't decrypt it (which is ok), but propagates decryption error and closes connection (which is wrong, it's fixed).

      Details: https://github.com/cossacklabs/acra/pull/305

      Source code(tar.gz)
      Source code(zip)
    • 0.84.0(Nov 9, 2018)

      0.84.0, November 9th 2018

      Core:

      • Key management

        • Improved LRU cache: fixed concurrent access to LRU cache by adding mutex. LRU cache is used for quick access to in-memory keys (private keys are stored encrypted) in AcraServer and AcraTranslator (#272).

        AcraServer documentation, AcraTranslator documentation.

        • Improved AcraRotate utility: added "dry-run" mode for testing AcraRotate before it is used for real. In the "dry-run" mode AcraRotate doesn't rotate keys: it fetches AcraStructs (from files or database), decrypts, rotates in-memory keys, encrypts the data with new public keys and prints the resulting JSON with new public keys without actually saving the rotated keys and AcraStructs. As key rotation might be tricky, we want users to make sure that AcraRotate has all the required permissions and access right before actually re-encrypting the data (#269).

        AcraRotate documentation.

      • AcraWriter

        • Added C++ AcraWriter library, added examples and tests. The library itself is a single header-only file acrawriter.hpp with dependency on Themis, placed in wrappers/cpp. Read the usage guide and examples in examples/cpp folder (#270).

        AcraWriter C++ documentation.

      • Logging

        • Improved logs of AcraConnector and AcraServer: use Debug log level for all network errors (closed connection, unavailable network, etc) and use Error log level only for cases of certainly unexpected behavior (#275).

        • Improved startup logs: log process PID on start of AcraServer, AcraConnector, AcraTranslator, and AcraWebConfig (#275).

        • Fixed timestamps: do not overwrite logs' timestamps (#273).

      • Tracing with OpenCensus

        • Added tracing with OpenCensus: AcraServer, AcraConnector, and AcraTranslator track every request from client application to the database and back. Each client request has a unique traceID that helps measure how much time it needs to perform a certain data processing functions (i.e. checking requests via AcraCensor, encrypting data, decrypting AcraStructs, etc.). Traces can be exported to Jaeger (#279, #276, #274).

        You can read more about tracing in our documentation in Tracing in Acra.

        A blogpost about technical details, profits, and pitfalls during the implementation of traces is coming soon.

      • Other

        • Improved AcraServer's connection handling: stop accepting connections after error and stop AcraServer instead of trying to accept connections after the listening socket was closed (#275).

        • Improved AcraCensor's handling of prepared statements for PostgreSQL binary protocol (#280).

        • Improved handling of terminating packets (COM_QUIT for PostgreSQL and TerminatePacket for MySQL) to correctly handle the closing connections from clients (#275).

        • Refactored inner logic of AcraCensor: improved code quality and stability, added more tests that use more patterns (#268).

      Infrastructure:

      • Ceased testing and supporting Go versions below 1.9. This will only affect the users who build Acra manually from sources. You can install the pre-built Acra components shipped for various Ubuntu, Debian, and CentOS distributives using Installation guide. Alternatively, you can check out our Docker images and Docker-compose files in docker folder (#277).

      • Tested Acra suite with PostgreSQL v11 and MariaDB v10.3 and updated docker compose examples and Acra Engineering Demo to use it (#278).

      • Published Acra load balancing demo: it illustrates some of the many possible variants of building high availability and balanced infrastructure based on Acra data protection suite components, PostgreSQL, and Python web application. In these examples we used HAProxy – one of the most popular high availability balancers today.

      • Updated AcraStruct Validator – an online tool that can decrypt your AcraStructs. AcraStruct Validator is useful for developers who build their own AcraWriters (to validate AcraStruct binary structure).

      Features coming soon:

      • Running SQL queries over encrypted data: perform AcraServer-side lookups (search) over protected data.

      • Pseudonymisation: early version of pseudonymisation library/plugin for Acra for transparent data pseudonymisation.

      • Cryptographically protected audit log: protection for logs against tampering.

      Documentation:

      • AcraWriter C++ has a short guide for installing and using AcraWriter for C++.

      • AcraRotate: added description and notes about "dry-run" mode.

      • Updated documentation for logging, collecting metrics, and tracing in Acra.

      • Many small fixes here and there to make your overall experience of using Acra's docs on a new platform distinctive and smooth ;)

      Source code(tar.gz)
      Source code(zip)
    • 0.83.0(Sep 28, 2018)

      0.83.0, September 28th 2018

      Core:

      • Security

        • Updated the default and allowed TLS configurations (#254).

          Use TLS v1.2 (the latest supported version in Golang now) and limited cipher suite recommended by Internet Assigned Numbers Authority and OWASP for most transport connections.

          Use TLS v1.0 and extended cipher suit for MySQL TLS connections due to the limited support of MySQL database and drivers.

        • Improved security of transport connection between Acra's services by validating the clientId length. This decreases the chance of misusing the clientId (#253).

      • Key management – key rotation

        • Added AcraRotate utility for rotation of Zone keys and re-encryption of AcraStructs. AcraRotate generates a new Zone keypair (zoneid_zone.pub and zoneid_zone) for a particular ZoneId and re-encrypts the corresponding AcraStructs with new keys. ZoneId stays the same (#256, #239).

          AcraRotate doesn't affect the ACRA_MASTER_KEY or storage keypair used without Zones (clientid_storage.pub / clientid_storage keys).

          AcraRotate rotates only the Zone storage keys and affects only the AcraStructs encrypted with Zones.

          AcraRotate works with AcraStructs stored both in database cells (MySQL or PostgreSQL) or files.

          Read the full documentation on AcraRotate on the Documentation Server.

      • AcraCensor – SQL filter and firewall

        • Improved SQL filtering through more complex pattern matching (#264, #263, #262, #238).

          • %%VALUE%% pattern represents literal value (string, binary, number, boolean) and is supported in the following expressions: WHERE, IN, ORDER BY, GROUP BY, BETWEEN.
          • %%LIST_OF_VALUES%% pattern represents several values one by one, used with IN expressions.
          • %%SUBQUERY%% pattern represents a subquery expression inside the main query.
          • %%WHERE%% pattern represents one or more expressions after a WHERE statement. This pattern works for SELECT/UPDATE/DELETE queries.
          • %%COLUMN%% pattern represents a column expression used after SELECT and ORDER BY expressions.
          • %%SELECT%% pattern represents a whole SELECT expression.

        Read the detailed description and usage examples on the AcraCensor page on DocServer.

      • AcraWriter

        • Added Java/Android AcraWriter library, added examples and tests (#252).

          Read the usage guide and examples in examples/android_java folder.

        • Added SQLAlchemy type wrappers for the Python AcraWriter (#257).

        • Improved and refactored the Python AcraWriter example of encrypting data and reading it from the database (#258).

      • Prometheus Metrics

        • Added functionality for exporting the basic metrics of AcraServer, AcraConnector, and AcraTranslator to Prometheus: if incoming_connection_prometheus_metrics_string is set, the service will generate specific metrics (time of connection life, time of processing requests, AcraStruct decryption counters) and push them to Prometheus (#260, #251, #234).
      • Other

        • Improved AcraConnector's compatibility with PostgreSQL: AcraConnector now correctly handles the database's denial to use TLS connection (#259).

        • Added export of CLI parameters for AcraServer, AcraConnector, and AcraTranslator to markdown (#261).

        • Improved readability of CEF-formatted logs by sorting extension fields in alphabetical order (#255).

        • Improved quality of our codebase — cleaned up the old unnecessary code (#250).

      Infrastructure:

      • Added AcraRotate as a ready-to-use tool inside AcraTranslator and AcraServer Docker containers (#236).

      Documentation:

      • Made the Documentation Server the primary and the only regularly updated source of documentation for Acra. The most recent version of the documentation, tutorials, and demos for Acra can be found there. The GitHub Wiki documentation for Acra is still available, but is no longer updated starting with the version 0.82.0 (with the exception of Changelog and README files with every new version release).

      • AcraCensor: updated the details on how the "patterns" filter works.

      • AcraRotate: added a tutorial for using AcraRotate to rotate Zone keys and re-encrypt the data.

      • Tons of small fixes here and there to make your overall experience of using Acra's docs on a new platform distinctive and smooth ;).

      Source code(tar.gz)
      Source code(zip)
    • 0.82.0(Aug 14, 2018)

      Core:

      • AcraTranslator

        AcraTranslator is a lightweight server that receives AcraStructs and returns the decrypted data. AcraTranslator doesn’t care about the source of the data, it accepts AcraStructs via HTTP or gRPC API. An application can store AcraStructs anywhere it is convenient: as cells in the database, as files in the file storage (local or cloud storage, like AWS S3). An application sends AcraStructs as binary data and receives plaintext (or decryption error) from AcraTranslator.

        However, sending plaintext data over a non-secure channel is a bad idea, so AcraTranslator requires the use of Themis Secure Session encryption channel (which is basically an encrypted TCP/unix sockets). To establish a Secure Session connection, an application doesn’t need to contain the crypto-code itself, only to direct the traffic through AcraConnector instead.

        AcraTranslator supports AcraStructs via HTTP and gRPC API, uses in-memory LRU cache to store encryption keys, and detects poison records. AcraTranslator is shipped as a pre-built binary and as a Docker container.

        Read the detailed guide on how to install, configure, and run AcraTranslator.

      (#213, #212, #207, #205, #204, #203, #200, #199, #198, #197)

      • Updated AcraConnector to support connections with AcraTranslator (#206).

      • Logging

        • Improved startup logs for AcraServer, AcraTranslator, and AcraConnector: now it's easier to understand that the service is up and running (#242).

        • Added clientId to AcraServer logs: now it's easier to understand which client was sending the request that led to a failed or successful AcraStruct decryption (#214).

        • Improved logging by masking query parameters: neither AcraServer nor AcraCensor won't leak sensitive query while logging the content (#216, #211).

      • Poison records

        • Poison record detection for AcraServer and AcraTranslator can now be turned on and off. Poison records detection is ON by default: AcraServer/AcraTranslator will try to detect poison record and log to stderr if a poison record is detected (#240, #230, #215).

        • Increased performance of AcraServer/AcraTranslator if poison records are enabled (#232).

      • Key management

        • Improved processing of decryption keys for AcraServer, AcraTranslator, and AcraConnector: now it is possible to store private keys encrypted in files and in memory, decrypt them before using, and purge after usage (#202).

        • Added configurable LRU cache for the in-memory keys: this will increase the performance if you operate with hundreds of storage keys on AcraServer and AcraTranslator (#219).

      • AcraCensor / SQL filter and firewall

        • Improved SQL filtering by adding pattern matching: now you can blacklist or whitelist queries that match particular patterns, like SELECT %%COLUMN%% FROM company %%WHERE%%.

        Currently supported patterns: %%VALUE%%, %%COLUMN%%, %%WHERE%% and %%SELECT%%.

        Read the detailed description and usage examples on the AcraCensor page.

        (#248, #247, #246, #245, #243, #238, #231, #226, #217)

        • Improved AcraCensor performance for queries matching (#208).
      • AcraWriter

        • Added iOS/Objective-C AcraWriter library as CocoaPod, added examples and tests.

        Read the usage guide and examples in examples/objc folder.

        (#241, #235, #233).

        • Improved security of AcraWriter libs for Go, Ruby, Nodejs, and iOS through zeroing secret keys where it was missing (#244).
      • AcraRollback

        • Improved handling of insert query parameter to simplify using AcraRollback from bash (#210).
      • Other

        • Improved AcraStruct decryption by multiple validations of AcraStruct format before decrypting. This fix improves AcraServer/AcraTranslator error messages in case of a failed decryption (#201).

        • Improved stability of integration test suite, trying to avoid 'timed out' errors from CircleCI (#200).

        • Improved code quality, fixing gofmt and golint issues (#229, #228, #227, #224, #223, #221).

      • WIP

        • Adding a way to export decryption metrics (decryption time, number of connections) from AcraServer to Prometeus (#234).

        • Prototyping AcraRotate tool for rotating the Zone keys easily (#239).

      Infrastructure:

      • Drop testing and supporting go version below 1.8. This will only affect the users who build Acra manually from sources. You can install pre-built Acra components shipped for various Ubuntu, Debian, and CentOS distributives using Installation guide. Alternatively, you can check out our Docker images and Docker-compose files in docker folder (#209).

      • Added AcraTranslator as pre-built binary and docker container (#222).

      • Added AcraTranslator and AcraConnector docker-compose files: now it's easy to set up a demo stand just by running one command (#225).

      • Added AcraRollback and AcraPoisonRecordMaker as ready-to-use tools inside AcraTranslator and AcraServer Docker containers (#236).

      Documentation:

      • Key management: clarified key names and default locations, illustrated public key exchange in details.

      • AcraServer: improved examples of how to run AcraServer.

      • AcraTranslator: added description of the service, installation and launching guide, added ready-to-use examples for HTTP and gRPC API.

      • AcraConnector: added examples of how to run AcraConnector with AcraTranslator.

      • AcraCensor: added examples of the configuration file, described in details how the "patterns" filter works.

      • AcraRollback: added examples of running AcraRollback in local mode, which allows storing the decrypted data locally, without pushing it back to the database.

      • This is the last version of Acra for which the main documentation will be actively updated in the GitHub Wiki. From now on, the most recent version of the documentation, tutorials, and demos for Acra will be available in the official Cossack Labs Documentation Server.

      Source code(tar.gz)
      Source code(zip)
    • 0.81.0(Jul 6, 2018)

      Core:

      • Prepared Statements

        • Added support of prepared statements for PostgreSQL/MySQL. Both binary and text response formats are supported (#192).
      • SQL requests filtering in AcraCensor

        AcraCensor got smarter in preventing SQL Injections.

        • Improved flexibility for parsing queries. If AcraCensor can't parse an SQL query, it is considered as potentially too dangerous to send it to the database, so AcraCensor blocks such "unparseable" queries by default. However, setting the configuration flag ignore_parse_error to true will make AcraCensor ignore the "unparseable" quality of queries and send them to the database anyway. Check out the configuration example in configs/acra-censor.example.yaml (#194).

        • Added support of complex JOIN queries (#191).

        • Improved reading/writing QueryCapture log file. Now AcraCensor uses bufferisation before writing queries into a log file. Changed format of QueryCapture log to JSON Line (each query sits in a separate line in a log file instead of having an array of JSON objects) (#193).

        • Introduced a few fixes here and there, made integration tests for AcraCensor more stable (#184).

      • Improving MySQL support

        We introduced MySQL support just a few Acra releases ago and we continue polishing it. Now we've updated the example projects so you can jump right into the code!

        Take a look at how to use Acra for both PostreSQL and MySQL databases in these examples:

      • Other

        • Updated handling of message formats for PostgreSQL and MySQL protocols (#186).

        • Improved logging in CEF and JSON formats for high load systems (#195).

        • Added comprehensive Readme to every project in /examples folder (#196).

        • Added pre-generated configuration file for AcraAuthmanager. Now it's easier to configure AcraServer using AcraWebconfig (#187).

      Documentation:

      Source code(tar.gz)
      Source code(zip)
    • 0.80.0(May 31, 2018)

      0.80.0, May 31st 2018

      Core:

      • Renaming

        • Global renaming of Acra components and their configuration parameters. We believe that the updated naming will decrease confusion about the components' functions and will make Acra's setup and usage process easier.

        Main services:

        | Old name | New name | Function | | --- | --- | --- | | AcraServer | AcraServer | decrypts data from the database | | AcraWriter | AcraWriter | encrypts data on the client side | | AcraProxy | AcraConnector | encrypts traffic between the client and the server using Themis Secure Session | | AcraCensor | AcraCensor | firewall, part of AcraServer, blocks suspicious SQL requests to the database | | AcraConfigUI | AcraWebConfig | lightweight HTTP web server for managing AcraServer's certain configuration options |

        Utilities:

        | Old name | New name | Function | | --- | --- | --- | | acra_rollback | AcraRollback | decrypts the whole database | | acra_genkeys | AcraKeymaker | generates encryption keys for storage and transport of the Acra components | | acra_genauth | AcraAuthmanager | generates user accounts for AcraWebConfig | | acra_genpoisonrecord | AcraPoisonRecordMaker | generates poision records for databases | | acra_addzone | AcraAddzone | generates Zones' header for AcraWriter |

        Check the configurations of components inside /configs folder and read Migration Guide for more details (#175, #174, #173, #170, #169, #168).

      • SSL/TLS

        • Improved SSL/TLS connections between AcraServer<->AcraConnector and AcraServer<->database. Added TLS authentication mode (tls_auth) argument to the AcraServer/AcraConnector configuration files:
          • for AcraConnector it indicates how to authenticate AcraServer during a TLS connection;
          • for AcraServer it indicates how to authenticate database during a TLS connection.
        • Updated TLS configuration to provide other less strict authentication methods (do not authenticate client from server, ask for any certificate, ask and check) (#171).
      • SQL requests filtering

        • Added support of filtering SQL requests for PostgreSQL databases. Now you can setup AcraCensor rules for both MySQL and PostgreSQL databases (#177).

        • Improved QueryCapture: AcraCensor writes allowed/blocked queries into a separate log file without blocking the main process (#176, #172).

        See a detailed description of AcraCensor on the corresponding AcraCensor documentation page.

      • AcraWriter in Ruby

      • Key Handling

        • Added make keys target in the Makefile: one command now generates keys and places them into correct folders for all Acra components (#182, #181).
        • Improved handling of master key length longer than 32 bytes (#183).
      • Other

        • Updated notification when AcraConnector is launched in an environment without netstat (#167).
        • Updated error handling for AcraServer working with Zones and fix some corner-cases in using PostgreSQL protocol (#186, #179).

      Infrastructure:

      • Even better Docker support

        • Added more ready-to-use Docker Containers: acra-keymaker, acra-authmanager. As a result, each Acra component is wrapped into a Docker container, allowing you to try Acra into your infrastructures easily.

        • Added easy-to-use docker-compose files for setting up the whole Acra-based environment connected to MySQL database. Possible configurations include setup with/without SSL, with/without AcraConnector, with/without Zones (#180). Check out the instructions and examples in the /docker folder: we have examples for both MySQL and PostgreSQL databases.

        • Updated descriptions for official Cossack Labs packages on Docker Hub.

        • Updated Getting started with Docker guide to make starting out with Acra even easier.

      • OS

        • Added support of Ubuntu Xenial, Ubuntu Bionic (added precompiled binaries and tests to make sure that Acra is compiling/building/working well on 16.04/18.04).

      Documentation:

      • Updated tutorials about protecting a Ruby on Rails app and a Django app.
      • Every single document, code line, and image are updated using the new naming.
      Source code(tar.gz)
      Source code(zip)
    • 0.77.0(Apr 13, 2018)

      Core:

      • MySQL databases

        • Added support for MySQL: now you can connect Acra to MySQL databases. Works with any SSL mode: require, allow, disable.
        • Tested and supported on: MySQL (#155, #140).

        Note: Prepared statements are not supported yet, but this feature is coming soon!

        Read about the new configurations on the AcraServer documentation page.

      • Keeping keys in secret

        • Added encryption for the keys' folder: private keys are now symmetrically encrypted by master_key (#143) for storage.
        • Added ability to generate public/private keys in the separate folders (#148, #142).

        Read more about the current changes in key management here.

      • Filtering requests for MySQL

        • Added firewall component named AcraCensor to handle MySQL queries.
          You can provide a list of restricted or allowed tables, columns, and exact queries to handle. AcraCensor will pass the allowed queries and return error on forbidden ones. Rules are configured and stored in yaml file. Each request is logged in real time. Moreover, all the queries and their states are logged into a separate log file. (#151, #138, #136, #132, #125, #108).

        See a detailed description of AcraCensor on the corresponding AcraCensor documentation page.

      • Web Config UI

        • Added lightweight HTTP web server for managing AcraServer's certain configuration options.
          You can update the proxy address and port, database address and port, handling of Zone mode and poison records. On saving new configuration, acraserver will gracefully restart and use these settings automatically. The access to thiw web page is restricted using basic auth. (#153, #141, #123, #111).

        See the interface screenshot and detailed instructions at Acra Config UI page.

      • Logging

        • Added support of new logging formats: plaintext, CEF, and json.
          Logging mode and verbosity level is configured for AcraServer, AcraProxy, and AcraConfigUI in the corresponding yaml files. Log messages were slightly improved, custom error codes were added (which we believe will help to understand and debug any issues) (#135, #126, #110).

        Read more about the log analysis at Logging page.

      • Tests

        • Added many new integartion tests, fixed stability and handling of more complicated use-cases (#150, #147, #137, #117, #116, #115).

      Infrastructure:

      • Docker support

        • Added Docker Container for every main component: AcraServer, AcraProxy, AcraConfigUI, and key generators (AcraGenKeys and AcraGenAuth). You can find the containers in /docker folder or on the Docker Hub (#139).

        • Updated Getting started with Docker guide to make starting out with Acra even easier.

        • Added easy-to-use docker-compose files to launch Acra in different environments, including key distribution. Possible configurations are:

          • acraserver + acra_configui;
          • connecting to PostreSQL or MySQL databases;
          • using Secure Session or SSL as transport encryption;
          • with or without acraproxy;
          • with or without zones.

          This is huge! We encourage you to try it! Check out the instructions and examples in the /docker folder. (#154, #146, #134, #133, #102).

      • Go versions

        • Updated the list of supported versions of Go. Every Acra component can now be built using Go >1.7, except acra_rollback that requires Go >1.8. No worries, you can still download Acra as a binary package anyway :)
      • OS

        • Dropped support of Debian Wheezy (no autotests, no precompiled binaries now).

      Documentation:

      • Updated QuickStart documentation about launching and building Acra components.
      • Described how to setup AcraCensor and AcraConfigUI.
      • Added more details and described new options (like using TLS and connecting to MySQL databases) for AcraServer and AcraProxy.
      • Described new logging formats.
      • Updated description of Key management approach we encourage you to use.
      • Described Docker components and ready-to-use Docker Compose configurations based on the Docker Readme.
      • Updated Getting started with Docker guide.
      • Distributed the information about master key across the docs.
      • Many small improvements.
      Source code(tar.gz)
      Source code(zip)
    • 0.76(Mar 9, 2018)

      Core:

      • SSL / TLS support

      Now you can use PostgeSQL with SSL/TLS settings enabled. Acra supports two modes of connection between AcraServer and the database: using SSL/TLS or using Secure Session (#113, #119).

      • Unix sockets

      Acra now supports usage of both TCP and Unix Sockets as a connection layer between AcraWriter <-> AcraProxy <-> AcraServer.

      • Tests

        • Updated integration test suit to support multiple connection modes between the Acra components and the database (#115, #117, #118, #120).
        • Added Docker image to make testing easier (#104).

      Infrastructure:

      • Added support of Go 1.10, removed support of older Go versions (<1.6).
      • Added support of Ubuntu 17.10, Ubuntu 16.04, Ubuntu 14.04, Debian Stretch.
      • Updated dependency libraries (libthemis and libcrypto) to use the latest ones.

      Documentation:

      • Updated the documentation and tutorials to reflect the latest changes.
      Source code(tar.gz)
      Source code(zip)
    • 0.75(Mar 7, 2017)

    Owner
    Cossack Labs
    convenient cryptographic tools where you need them
    Cossack Labs
    How to systematically secure anything: a repository about security engineering

    How to Secure Anything Security engineering is the discipline of building secure systems. Its lessons are not just applicable to computer security. In

    Veeral Patel 6.1k Jul 23, 2021
    Build Go applications for IOS

    go-build-for-ios Build Go applications for IOS This repository contains a PoC that lets you build any Go application for IOS platform. Cross-compilati

    Marcin Tojek 19 Mar 1, 2021
    Easy to use cryptographic framework for data protection: secure messaging with forward secrecy and secure data storage. Has unified APIs across 14 platforms.

    Themis provides strong, usable cryptography for busy people General purpose cryptographic library for storage and messaging for iOS (Swift, Obj-C), An

    Cossack Labs 1.3k Jul 16, 2021
    Agent-less vulnerability scanner for Linux, FreeBSD, Container, WordPress, Programming language libraries, Network devices

    Vuls: VULnerability Scanner Vulnerability scanner for Linux/FreeBSD, agent-less, written in Go. We have a slack team. Join slack team Twitter: @vuls_e

    Future Corp 8.6k Jul 25, 2021
    crowdsec 3.5k Jul 20, 2021
    DockerSlim (docker-slim): Don't change anything in your Docker container image and minify it by up to 30x (and for compiled languages even more) making it secure too! (free and open source)

    Minify and Secure Docker containers (free and open source!) Don't change anything in your Docker container image and minify it by up to 30x making it

    docker-slim 10.3k Jul 24, 2021
    Ah shhgit! Find secrets in your code. Secrets detection for your GitHub, GitLab and Bitbucket repositories: www.shhgit.com

    shhgit helps secure forward-thinking development, operations, and security teams by finding secrets across their code before it leads to a security br

    Paul 3.2k Jul 22, 2021
    An easy-to-use XChaCha20-encryption wrapper for io.ReadWriteCloser (even lossy UDP) using ECDH key exchange algorithm, ED25519 signatures and Blake3+Poly1305 checksums/message-authentication for Go (golang). Also a multiplexer.

    Quick start Prepare keys (on both sides): [ -f ~/.ssh/id_ed25519 ] && [ -f ~/.ssh/id_ed25519.pub ] || ssh-keygen -t ed25519 scp ~/.ssh/id_ed25519.pub

    null 18 Jul 19, 2021
    A tool for secrets management, encryption as a service, and privileged access management

    Vault Please note: We take Vault's security and our users' trust very seriously. If you believe you have found a security issue in Vault, please respo

    HashiCorp 21.5k Jul 20, 2021
    A collection of cool tools used by Mobile hackers. Happy hacking , Happy bug-hunting

    A collection of cool tools used by Mobile hackers. Happy hacking , Happy bug-hunting Family project Table of Contents Weapons Contribute Thanks to con

    HAHWUL 241 Jul 15, 2021
    Automatic HTTPS for any Go program: fully-managed TLS certificate issuance and renewal

    Easy and Powerful TLS Automation The same library used by the Caddy Web Server Caddy's automagic TLS features—now for your own Go programs—in one powe

    Caddy 3.7k Jul 24, 2021
    Nuclei is a fast tool for configurable targeted vulnerability scanning based on templates offering massive extensibility and ease of use.

    Fast and customisable vulnerability scanner based on simple YAML based DSL. How • Install • For Security Engineers • For Developers • Documentation •

    ProjectDiscovery 4.7k Jul 23, 2021
    Container Signing

    cosign Container Signing, Verification and Storage in an OCI registry. Cosign aims to make signatures invisible infrastructure. Info Cosign is develop

    sigstore 717 Jul 23, 2021
    A simple, modern and secure encryption tool (and Go library) with small explicit keys, no config options, and UNIX-style composability.

    age age is a simple, modern and secure file encryption tool, format, and library. It features small explicit keys, no config options, and UNIX-style c

    Filippo Valsorda 6.4k Jul 21, 2021