Database security suite. Database proxy with field-level encryption, search through encrypted data, SQL injections prevention, intrusion detection, honeypots. Supports client-side and proxy-side ("transparent") encryption. SQL, NoSQL.

Last update: Dec 28, 2022

Overview

Database protection suite with field level encryption and intrusion detection.

| Acra Engineering Examples | Documentation | Installation | Examples and tutorials | | ---- | ---- | ---- | --- | --- |

What is Acra

Acra — database security suite for sensitive and personal data protection.

Acra provides field level encryption (client-side or proxy-side), multi-layered access control, database leakage prevention, and intrusion detection capabilities in a convenient, developer-friendly package. Acra was specifically designed for web and mobile apps with centralised data storage, including with distributed, microservice-rich applications.

Perfect Acra-compatible applications	Typical industries
Web and mobile apps that store data in a centralised database or object storage	Healthcare Finance E-commerce Critical infrastructures Apps with > 1000 users
IoT apps that collect telemetry and process data in the cloud
High-load data processing apps

Acra gives you tools for encrypting the data records into special cryptographic containers, storing them in the database or file storage and then decrypting them in a secure compartmented area (separate virtual machine/container). Acra allows to encrypt data as early as possible and operate on encrypted data.

Acra's cryptographic design ensures that no secret (password, key, etc.) leaked from the application or database will be sufficient for decryption of the protected data chunks that originate from it. Acra minimises the leakage scope, detects unauthorised behavior, and prevents the leakage, informing operators of the incident underway.

This is Acra Community Edition, it's free for commercial and non-commercial use, forever.

Major security features

Cryptographic protection of data	field level encryption (client-side and/or proxy-side); each data record is encrypted using unique encryption keys
Selective encryption	select which field/columns to encrypt to balance good security and performance
Key management tools	built-in tools for key distribution, key rotation (with/without data re-encryption), and compartmentalisation
Trust compartmentalisation	datastore and application components can be compromised, yet the data stays protected
Prevention of SQL injections	through a built-in SQL firewall
Intrusion detection system	using poison records (honey tokens) to warn about suspicious behaviour
REST API (gRPC, HTTP)	encryption/decryption using AcraTranslator
Searchable encryption ᵉ	available for Acra Enterprise users
Masking / anonymization ᵉ
Tokenization / Pseudonymization ᵉ
Cryptographically protected audit log ᵉ

Acra delivers different layers of defense for different parts and stages of the data lifecycle. This is what defence in depth is – an independent set of security controls aimed at mitigating multiple risks in case of an attacker crossing the outer perimeter.

Developer and DevOps friendly

Secure default settings	your infrastructure is secure from the start without additional configuring
Cryptography is hidden under the hood	no risk of selecting the wrong key length or algorithm padding
Automation-friendly	easy to configure and automate
Quick infrastructure integration	via binary packages or Docker images
Easy client-side integration	client-side encryption libraries support ~11 languages
Code-less client-side integration	available for server-side encryption in AcraServer's Transparent proxy mode
Logging, metrics, tracing	throughout all Acra components; compatible with ELK stack, Prometheus, Jaeger
No vendor lock	rollback utilities to decrypt database into plaintext
Demos, examples, simulators	numerous web-based and Docker-based example projects available
DigitalOcean Acra 1-Click App	run AcraServer in your DigitalOcean cloud
Managed solution available	we can setup and manage Acra for you

Cryptography

Acra relies on our cryptographic library Themis, which implements high-level cryptosystems based on the best available open-source implementations of the most reliable ciphers. Acra strictly doesn't contain self-made cryptographic primitives or obscure ciphers. To deliver its unique guarantees, Acra relies on the combination of well-known ciphers and smart key management scheme.

Default crypto-primitive source	OpenSSL
Supported crypto-primitive sources ᵉ	BoringSSL, LibreSSL, FIPS-compliant, GOST-compliant, HSM
Storage encryption	AES-256-GCM + ECDH
Transport encryption	TLS v1.2+ / Themis Secure Session
KMS integration ᵉ	Amazon KMS, Google Cloud Platform KMS, Hashicorp Vault, Keywhiz

ᵉ — available in the Enterprise version of Acra only. Drop us an email to get a full list of features and a quote.

Try Acra without writing code

How does Acra work?

To better understand the architecture and data flow in Acra, please refer to the Architecture and data flow section in the documentation.

Protecting data in SQL databases using AcraServer

AcraServer works as transparent encryption/decryption proxy with SQL databases. Depending on your app architecture, it's possible to use client-side encryption or server-side encryption, or both simultaneously.

Integrating client-side encryption using AcraWriter

The typical infrastructure with AcraConnector and AcraServer, data is encrypted on the client side (web backend or mobile app) using AcraWriter, decrypted on the AcraServer side, that works as transparent database proxy.

The dataflow of encrypted and plaintext data. Note: it's possible to write encrypted data directly to the database, omitting the AcraServer.

This is what the process of encryption and decryption of data in a database looks like:

Your application encrypts some data through AcraWriter by generating an AcraStruct using Acra storage public key and then updates the database. AcraStructs generated by AcraWriter can't be decrypted by it — only the Acra's server side has the keys for decryption.
To retrieve the decrypted data, your application talks to AcraServer. It is a server-side service that works as a database proxy: it transparently sits between your application and the database and silently listens to all the traffic coming to and from the database.
AcraServer monitors the incoming SQL requests and blocks the unwanted ones using the built-in configurable SQL firewall called AcraCensor. AcraServer only sends allowed requests to the database. Certain configurations for AcraServer can be adjusted remotely using AcraWebConfig web server.
Upon receiving the database response, AcraServer tries to detect AcraStructs, decrypts them, and returns the decrypted data to the application.
AcraConnector is a client-side daemon responsible for providing encrypted and authenticated connection between the application and AcraServer. AcraConnector runs under a separate user/in a separate container and acts as middleware. AcraConnector accepts connections from the application, adds an extra transport encryption layer using TLS or Themis Secure Session, sends the data to AcraServer, receives the result, and sends it back to the application.

Integrating proxy-side encryption using AcraServer in Transparent proxy mode

The typical infrastructure with AcraConnector and AcraServer, data is encrypted and decrypted by AcraServer, that works as transparent database proxy.

The dataflow of encrypted and plaintext data, AcraServer it encrypts and decrypts data.

Transparent proxy mode allows you to configure AcraServer to encrypt records in specific database columns without altering the application code. Basically, AcraServer here performs AcraWriter's duties.

This mode is useful for large distributed applications, where separate updating of the source code for each client app is complicated.

Imagine that the client application wants to update some data in the database. The application logic doesn't need to change because there's no need to integrate AcraWriter library into the application code. The application sends SQL requests with plaintext data through AcraConnector and AcraServer to the database, as described above. There's a secure transport layer between AcraConnector and AcraServer (via TLS or Themis Secure Session), so data from client app is transmitted via a secure channel.
On receiving SQL queries, AcraServer parses each query and encrypts the desired values into AcraStructs. To know which values to encrypt, AcraServer uses a special configuration file (known as encryptor_config_file) that describes which columns in which tables should be encrypted. AcraServer uses AcraStruct encryption key just as AcraWriter would.
After encrypting the values into AcraStructs, AcraServer passes the modified queries to the database and the database response – back to the client application via AcraConnector.
When the client application wants to read the data, it sends a read query to the database (via AcraConnector and AcraServer). Upon retrieving the database response, AcraServer tries to detect AcraStructs, decrypts them, and returns the decrypted data to the application.

Transparent mode has the same features as the "usual" mode – including SQL firewall, intrusion detection based on poison records, Zone support, etc.

Check out the detailed documentation on using and configuring the Transparent proxy mode.

Protecting data in any file storage using AcraWriter and AcraTranslator

The typical infrastructure with AcraConnector and AcraTranslator, data is encrypted on the client side (web backend or mobile app) using AcraWriter, decrypted on the AcraTranslator side, that works as standalone decryption service.

The dataflow of encrypted and plaintext data, AcraTranslator only decrypts data on demand.

In some use cases, the application can store encrypted data as separate blobs (in NoSQL databases, or as files that are not in a database, i.e. in a S3 bucket, local file storage, etc.). In this case, you can use AcraTranslator — a lightweight server that receives AcraStructs and returns the decrypted data.

Note: since Q3 2020, AcraTranslator can both encrypt and decrypt data via REST API (HTTP, gRPC). You may use a combination of AcraWriter+AcraTranslator for client-side encryption and Translator-side decryption; or AcraTranslator only for both encryption and decryption without changing app code.

This is what the process of encryption and decryption of data using AcraTranslator looks like:

Your application encrypts some data using AcraWriter, generating an AcraStruct using Acra storage public key and puts the data into any file storage. AcraStructs generated by AcraWriter can't be decrypted by it — only the Acra's server side has the right keys for decrypting it. Otherwise, you might decide not to use AcraWriter, but use AcraTranslator for encrypting data – application sends plaintext, AcraTranslator returns AcraStructs.
To decrypt an AcraStruct, your application sends it to AcraTranslator as a binary blob via HTTP or gRPC API. AcraTranslator doesn’t care about the source of the data, it is responsible for holding all the secrets required for data decryption and for actually decrypting the data.
AcraTranslator decrypts AcraStructs and returns the decrypted data to the application.
To avoid sending plaintext via an unsecured channel, AcraTranslator requires the use of AcraConnector, a client-side daemon responsible for providing encrypted and authenticated connection between the application and AcraServer. AcraConnector runs under a separate user/in a separate container and acts as middleware. It accepts connections from the application, adds transport encryption layer using TLS or Themis Secure Session, sends data to AcraServer, receives the result, and sends it back to the application.

AcraTranslator and AcraServer are fully independent server-side components and can be used together or separately depending on your infrastructure.

Availability

Client-side

AcraWriter is a client-side library that encrypts data into a special binary format called AcraStruct. AcraWriter is available for Ruby, Python, Go, C++, Node.js, iOS, Android/Java and PHP, but you can easily generate AcraStruct containers with Themis for any platform you want.

AcraWriter is required only for the client-side encryption dataflow. If you use AcraServer in Transparent encryption mode, or use AcraTranslator for both encryption and decryption, you don't need AcraWriter at all.

Client platform	Documentation and guides	Examples
🐹 Go	Installation guide	examples/golang
🐍 Python	Installation guide	examples/python
♦️ Ruby	Installation guide	examples/ruby
➕ C++	Installation guide	examples/cpp
📱 iOS (Swift / Objective-C)	Installation guide	examples/objc
☎️ Android (Java / Kotlin)	Installation guide	examples/android_java
🐘 PHP	Installation guide	examples/php
🍭 Node.js	Installation guide	examples/nodejs

Server-side

The Server-side Acra components (AcraServer, AcraTranslator) should run as separate services/servers/VMs. Key management utils and database helpers are usually run on the same server as AcraServer/AcraTranslator.

There are several possible ways to install and launch Acra components in your infrastructures:
- download and run our Docker-based demo stand to deploy everything you need using a single command.
- download pre-built Acra binaries for supported distributives.
- install AcraServer droplet on DigitalOcean, configure manually or use pre-defined interactive configuration script.
- build from sources (Acra is built and tested with Go versions 1.13 – 1.15).
- talk to us, if you would like to run managed Acra (Acra-as-a-Service).

Install Server-side Acra components:

Distributive	Instruction set	Download package	or alternatively
CentOS 7	x86_64	rpm package	use Docker or build from source
Debian Stretch (9) Debian Jessie (8)	x86_64/i386	deb package
Ubuntu Bionic (18.04)	x86_64	deb package
Ubuntu Artful (17.10) Ubuntu Xenial (16.04)	x86_64/i386	deb package
macOS (10.12 - 10.14)	x86_64	-

Note: the server-side Acra components are not compatible with Windows OS as host OS, consider using Docker.

Compatibility and integration

AcraServer is a server-side service that works as database proxy: it sits transparently between your application and the database, listens silently to all the traffic that's coming to and from the database. AcraTranslator is database-agnostic: it provides HTTP and gRPC API to decrypt AcraStructs stored anywhere.

Acra is compatible with numerous RDBMS, object and KV stores, cloud platforms, external key management systems (KMS), load balancing systems.

Cloud platforms	DigitalOcean, AWS, GCP, Heroku
RDBMS	MySQL v5.7+, PosgtreSQL v9.4-v11, MariaDB v10.3 Google Cloud SQL, Amazon RDS
Object stores	filesystems, KV databases, Amazon S3, Google Cloud DataStore
Load balancing	HAProxy, cloud balancers

Open source Acra has limited integration support, more services are available in the Enterprise version of Acra only.

Installation and launch

Quick try (run example apps)

Acra Example Projects illustrate the integration of Acra data protection suite into existing applications: web applications based on Django and Ruby on Rails frameworks, and simple CLI applications. We took well-known apps, detected sensitive data there and added the encryption layer. Protecting the data is completely transparent for the users and requires minimal changes in the infrastructure and application code.

Developers and Ops friendly:

run a single command to deploy the application, database, Acra's components, logs, and dashboards;
read the code changes and see how little it takes to integrate encryption into the client application;
learn how Acra works by reading logs, monitoring metrics in Prometheus, checking tracers in Jaeger and watching Grafana dashboards;
inspect Docker-compose files, architecture schemes, database tables, and much more.

Requirements: Linux or macOS with installed Docker.

⚙️ Run Acra Example Projects ⚙️

Quick integration into your infrastructure

For a quick and easy integration of Acra into your own infrastructure, we recommend trying Acra with Docker first. Using only two commands, you will get all the Acra's components and database up and running, with a secure transport layer between them. We prepared several typical infrastructure variants to experiment with.

Select one appropriate use case from the pre-made configurations ("Compose files"): use AcraServer-based configuration to protect the data in a database or select AcraTranslator to protect the files or any other binary blob stored elsewhere.
Launch Acra's server-side by running the selected Docker-compose file: it will generate the appropriate keys, put them into correct folders, perform a public key exchange, run selected services and database, and then it will listen to the incoming connections.
Integrate AcraWriter into your application code where you need to protect the sensitive data, supply AcraWriter with an Acra storage public key (generated by Docker-compose on the previous step). Encrypt the data into AcraStructs and send them into the database or file storage.
Decrypt data by reading the database through AcraServer or by decrypting the files through AcraTranslator.

Please use the Acra Docker demo stand for testing/experimenting purposes only as the encryption keys are pre-generated in the configuration.

Normal integration into your infrastructure

For production environments, we insist on generating and exchanging keys manually and deploying Acra as Docker containers or from source code. Refer to the Quick Start guide to understand how to download and launch Acra components, generate keys, and perform the key exchange properly.

Documentation, tutorials, additional information

The most recent versions of the documentation, tutorials, and demos for Acra are available on the official Cossack Labs Documentation Server. The Github Wiki documentation is deprecated and is no longer updated since v0.82.0.

To gain an initial understanding of Acra, you might want to:

Read about using the lightweight HTTP web server AcraWebConfig we provide to manage AcraServer configuration in a simple fashion.
Read the notes on security design and intrusion detection to better understand what you get when you use Acra and what is the threat model that Acra operates in.
Key and trust management tools: key distribution, key rotation and database rollback.
Set up rules for AcraCensor (SQL firewall) suitable for your application.
Read some notes on making Acra stronger, more productive, and efficient and about adding security features or increasing throughput, depending on your goals and security model.
Read about the logging format that Acra supports if you are using a SIEM system.

You can also check out the speaker slides for the following talks by Cossack Labs engineers:

Demo projects

Client-side encryption mode	Transparent encryption mode

AcraCensor: SQL firewall to prevent SQL injections

AcraCensor example application illustrates how to use AcraCensor as SQL firewall to prevent SQL injections. The target application is a well-known vulnerable web application OWASP Mutillidae 2.

The demo project has a Docker-compose file that runs the following web infrastructure:

OWASP Mutillidae web application,
Acra encryption suite.

Acra works as a proxy between web and database. AcraCensor inspects every SQL query that runs from the web application to the database, and back, and blocks suspicious queries.

Requirements: Linux or macOS with installed Docker.

⚙️ Run AcraCensor SQL firewall example ⚙️

Poison records: intrusion detection example

Intrusion detection example application illustrates how to use poison records (honey tokens) as "intrusion markers" that warn about possible attack.

Requirements: Linux or macOS with installed Docker.

🍯 Run intrusion detection example 🐝

Acra Load Balancing Example

Acra load balancing example application illustrates building high availability and balanced infrastructure, based on Acra components, PostgreSQL, and Python web application. We prepared several configurations with multiple databases and HAProxy.

🔛 Run Acra Load Balancing Example 🔛

GDPR, HIPAA, CCPA

Acra can help you comply with the current privacy regulations, such as:

Configuring and using Acra in a designated form will cover most of the demands described in articles 25, 32, 33, and 34 of GDPR and the PII data protection demands of HIPAA. Read more about Acra and GDPR compliance here.

Open source vs Pro vs Enterprise

This is Acra Community Edition, the open source version of Acra, which is 💯 free for commercial and non-commercial usage. Please let us know in the Issues if you stumble upon a bug, see a possible enhancement, or have a comment on security design.

There are also Pro and Enterprise versions of Acra available. Those versions provide better performance, redundancy/load balancing, come pre-configured with crypto-primitives of your choice (FIPS, GOST), have integration with key/secret management tools in your stack, provide unique security features like search through encrypted data, and have plenty of utils and tools for your Ops and SREs to operate Acra conveniently – deployment automation, scaling, monitoring, and logging. Talk to us to get full feature lists and a quote.

Security consulting

It takes more than just getting cryptographic code to compile to secure the sensitive data. Acra won't make you “compliant out of the box” with all the modern security regulations, and no other tool will.

We help companies plan their data security strategy by auditing, assessing data flow, and classifying the data, enumerating the risks. We do the hardest, least-attended part of reaching the compliance – turning it from the “cost of doing business” into the “security framework that prevents risks”.

Contributing to us

If you’d like to contribute your code or provide any other kind of input to Acra, you’re very welcome. Your starting point for contributing is here.

License

Acra is licensed as Apache 2 open-source software.

Contacts

If you want to ask a technical question, feel free to raise an Issue or write to [email protected].

To talk to the business wing of Cossack Labs Limited, drop us an email to [email protected].

Comments

Support for AcraServer integration with Patroni HA

Hey folks, feature request/question here.

We are currently using patroni as an HA solution for PostgreSQL and want to use Acra as an encryption/decryption DB proxy for on-the-fly transparent data encryption.

In Patroni, a. HAproxy connects with multiple postgres nodes, based on a leader-replica architecture using raft.

Is it possible to integrate AcraServer over the same patroni's HAProxy, so that we can introduce AcraServer without sacrificing our DB's latency instead of a separate AcraServer instance?
question

opened by kumar1202 23
Adding data type to encrypted column in AcraServer encryption configuration
issues:

In the documentation, this is mentioned :

Transparent encryption proxy mode allows you to configure AcraServer to encrypt records in specific database columns without altering the application code.

However, There is not possibility for my not editable application to work with this proxy as :

Encrypted data is binary data. As AcraServer doesn't know the nature of data, it returns the decrypted binary data to the web app. You’ll need to change the source code of your web app for the app to expect the decrypted data as binary, then to encode it into the original format (strings, numbers, bytes, etc.).

Expected behaviour :

As a transparent proxy user , I expect to have to define the database field type either in the acra server/connector configuration and in the database (change encrypted field into binary field) but not in the application code.

Regards,
enhancement acra-server feature-request
opened by wdesplas 15

AcraServer 0.93 + PostgreSQL | "Panic in connection processing"

Hello, I have been testing Acra Server 0.92.0 with an application. Version 0.92.0 seems to be working fine. I then realized I needed some features present in the 0.93.0 release but after updating I can't no longer run my application with acra.

To Reproduce

Set up acra server 0.92.0 with my application
Updated Acra server from 0.92.0 to 0.93.0
Application no longer works with acra server

Expected behavior Application should work as before

Acra configuration files

--client_id="test"
--db_host=postgres
--keys_dir=/keys
--tls_auth=0
--tls_client_id_from_cert=false
--postgresql_enable=true
-d

Environment (please complete the following information):

Acra version: 0.93.0
Database server and its version: PostgreSQL 9.6.10
Installed components:
- [x] AcraServer
Data-in-transit encryption between Acra and the client-side application:
- [x] no transport encryption
Installation way:
- [x] via Docker

Additional context Acra server debug logs when the application crashes

time="2022-09-01T14:04:32Z" level=debug msg="New query" client_id=test prepared_name=S_4 proxy=client session_id=1 sql=begin
time="2022-09-01T14:04:32Z" level=debug msg="Registered new prepared statement" client_id=test prepared_name=S_4 proxy=client session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Read data" client_id=test proxy=client session_id=1
time="2022-09-01T14:04:32Z" level=debug msg=GetBindData client_id=test proxy=client session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Bind packet" client_id=test portal="" proxy=client session_id=1 statement=S_4
time="2022-09-01T14:04:32Z" level=debug msg="Read data" client_id=test proxy=client session_id=1
time="2022-09-01T14:04:32Z" level=debug msg=GetExecuteData client_id=test proxy=client session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Read data" client_id=test proxy=client session_id=1
time="2022-09-01T14:04:32Z" level=debug msg=GetParseData client_id=test proxy=client session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="New query" client_id=test prepared_name=S_5 proxy=client session_id=1 sql="insert into blue_FDZ_CONFIGURATION(\"key\", value) values ($1, $2)"
time="2022-09-01T14:04:32Z" level=debug msg="Hasn't schema for table blue_FDZ_CONFIGURATION"
time="2022-09-01T14:04:32Z" level=debug msg="Registered new prepared statement" client_id=test prepared_name=S_5 proxy=client session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Read data" client_id=test proxy=client session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Read data" client_id=test proxy=client session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Read data length" client_id=test proxy=server session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Read data" client_id=test proxy=server session_id=1
time="2022-09-01T14:04:32Z" level=debug msg=ParseComplete parse="&{[83 95 53 0] [73 78 83 69 82 84 32 73 78 84 79 32 34 98 108 117 101 95 70 68 90 95 67 79 78 70 73 71 85 82 65 84 73 79 78 34 32 40 34 107 101 121 34 44 32 34 118 97 108 117 101 34 41 32 86 65 76 85 69 83 32 40 36 49 44 32 36 50 41 0] [0 2] [[0 0 4 19] [0 0 4 19]]}"
time="2022-09-01T14:04:32Z" level=debug msg="Read packet" client_id=test proxy=server session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Read data length" client_id=test proxy=server session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Read data" client_id=test proxy=server session_id=1
time="2022-09-01T14:04:32Z" level=error msg="Panic in connection processing, close connection" client_id=test error="runtime error: invalid memory address or nil pointer dereference" function=ProxyDatabaseConnection session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Close acra-connector connection" client_id=test session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Can't read first 5 bytes" client_id=test code=588 error="read tcp 172.30.0.8:9393->172.30.0.9:56294: use of closed network connection" proxy=client session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Can't read packet from client to database" client_id=test error="read tcp 172.30.0.8:9393->172.30.0.9:56294: use of closed network connection" proxy=client session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Stop to proxy" client_id=test interrupt_side=Client-AcraServer session_id=1
time="2022-09-01T14:04:32Z" level=error msg="Network error" client_id=test code=1100 error="read tcp 172.30.0.8:9393->172.30.0.9:56294: use of closed network connection" interrupt_side=Client-AcraServer session_id=1
time="2022-09-01T14:04:32Z" level=info msg="Closing client's connection" client_id=test interrupt_side=Client-AcraServer session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Close acra-connector connection" client_id=test session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Close db connection" client_id=test session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="Close db connection" client_id=test session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="All connections closed" client_id=test session_id=1
time="2022-09-01T14:04:32Z" level=debug msg="All connections closed" client_id=test session_id=1
time="2022-09-01T14:04:33Z" level=debug msg="Can't read first 5 bytes" client_id=test code=588 error=EOF proxy=client session_id=2
time="2022-09-01T14:04:33Z" level=debug msg="Can't read packet from client to database" client_id=test error=EOF proxy=client session_id=2
time="2022-09-01T14:04:33Z" level=debug msg="Stop to proxy" client_id=test interrupt_side=Client-AcraServer session_id=2
time="2022-09-01T14:04:33Z" level=debug msg="EOF connection closed" client_id=test interrupt_side=Client-AcraServer session_id=2
time="2022-09-01T14:04:33Z" level=info msg="Closing client's connection" client_id=test interrupt_side=Client-AcraServer session_id=2
time="2022-09-01T14:04:33Z" level=debug msg="Close acra-connector connection" client_id=test session_id=2
time="2022-09-01T14:04:33Z" level=debug msg="Close db connection" client_id=test session_id=2
time="2022-09-01T14:04:33Z" level=debug msg="Can't read packet" client_id=test code=588 error="read tcp 172.30.0.8:35104->172.30.0.4:5432: use of closed network connection" proxy=server session_id=2
time="2022-09-01T14:04:33Z" level=debug msg="All connections closed" client_id=test session_id=2
time="2022-09-01T14:04:33Z" level=debug msg="Second proxy goroutine stopped" client_id=test error="AcraServer-Database:read tcp 172.30.0.8:35104->172.30.0.4:5432: use of closed network connection" interrupt_side=Client-AcraServer session_id=2
time="2022-09-01T14:04:33Z" level=info msg="Finished processing client's connection" client_id=test interrupt_side=Client-AcraServer session_id=2

Also tried to run different commits of release 0.93.0 and this starts to happen after commit 29eb6ffb20ecbefd2f7f1684ec16666c1ab6169a

bug acra-server postgresql

opened by Machado117 11

Add new linters to Acra CircleCI configuration
Fix golint not being installed in prepare.sh and thus not reporting issues

Add misspell linter to detect common misspells

Add ineffassign linter to detect ineffectual assignments (assigned but not used or unintentionally assigned to wrong var because of how scopes and := work in Go)

Make all these linters write output to console, so we'll see what the issues are by looking at CI log

I'm still unsure whether it's good idea to allow 10 ineffassign issues. There are currently 3, and those are intentional = nil assignments. Like, we won't see them until we reach 10, but then we won't be able to merge PR because CircleCI tells us there're too many issues
infrastructure docker
opened by iamnotacake 11
Question about Searchable Encryption

Hi,

I am looking for a tutorial that's gonna make me test your searchable encryption part of software but the one in the examples is empty and "coming soon".

I have read the provided guide anyway and I cannot find the "encryptor_config_file" in the acra engineering demo. How I do?

Cheers.
question

opened by soluzioninformatiche 9
TLS authentication with certificates as identifiers
Extended current TLS authentication where acra may starts with specified from cli params client_id which used for all incoming connections. This PR extends it and allow to start acra without static client_id and use data from certificates to map them to keystore identifiers/keys. PR contains 2 implementations of extracting data from certificates: common name and serial numbers, which should be chosen at startup (cli params not added yet). And one implementation of converting these values to acceptable format of ids using hex encoder (to allow non ascii printable values like serial numbers) and hash function (to allow values which longer than 256 length). A little bit refactored unit-tests to re-use same common function which verify wrapper's logic, added helpers to generate tls certificates in golang to test all specific cases related to certificate fields (@iamnotacake you can re-use them if need^)

Done:

validation incoming certificate (deny CA certificates or with incorrect keyUsage/extKeyUsage values)

identifier fetching from certificates (distinguished name, serial numbers)

encoding identifier values to acceptable formats of clientID by sha512 and hex encoding

Now AcraServer can use certificates from acra-connector connections when used tls_client_id_from_cert + acraconnector_tls_transport_enable, or from app's connection when used tls_client_id_from_cert + acraconnector_transport_encryption_disable. In both cases should be configured all necessary TLS params with ca/cert/key values. Plus user can configure which strategy of mapping certificate metadata to clientID to use. To configure should set tls_identifier_extractor_type with distinguished_name or serial_number value. In first case will be used "Subject" field ordered and formatted according to RFC2253, hashed with sha512 (as hash function which provide longest digest and least collision probability) and encoded to hex value. In second cases will be used serial number as 20-bytes integer value in big-endian order hashed with same function sha512 and encoded to hex value. First strategy good to use when certificates may be rotated and allow to avoid rotation keys and accordingly data too. Second strategy good for environments when Subject field not used in certificates.
opened by Lagovas 9
AcraTranslator refactoring: service start up and shutdown, signals listener
This PR introduces several updates for our AcraTranslator service.

In general I refactored main component (ReaderServer) and added support of SIGHUP handling to the service. What is refactored:

Added new StartFromFileDescriptor function that allows to run HTTP and gRPC servers having listener obtained from correspondent descriptor, instead of creating a new ones in order to preserve current listeners in new process;

Splitted monster Start function on lower components startHTTP, startGRPC in order to improve readability;

Wrapped Start and StartFromFileDescriptor with timeout exit (single point) and avoiding hidden os.Exit calls. This is extremely important for EE version that relies on CE ReaderServer component;

Moved from 'multiply listeners - multiply connections' to 'single listener - multiply connection' with transparent management of stopping/closing it. This is also important for better controlling of code execution;

Added some getters which is needed by EE version.

There are two questions about previous implementation that were not obvious for me (most likely they are addressed to @Lagovas), and probably we can discuss it while review process: 1) why previously we had 'multiply listeners - multiply connection' model (when we called HandleClientConnection, it was a AcceptConnection function inside that created internally new listener and returned accepted connection via channel, their closing was managed via child contexts cancelling in separate goroutine) if listener itself allows calling Accept function safely from multiply goroutines; and 2) why does switching to new process works, if file descriptors created while SIGHUP processing and on service starting are different? (I asked @Lagovas about it and we came to the point that we know order number of Acra descriptor - it's 3, but it seems that we need not the order number but correct actual value of fd to start)

How did I test this? Manually. I created a following methodology for testing it (locally run AcraTranslator, AcraConnector and 2 toy applications: one for sending HTTP and second - for gRPC requests):

Test HTTP component without/with active connections;

Test gRPC component without/with active connections;

Test HTTP + gRPC components that run simultaneously without/with active connections;

For sending HTTP requests I used the following piece of code (thanks to our documentation - but some examples are outdated there):

for run in {1..100}; do curl -X POST --data-binary @client.acrastruct --header "Content-Type: application/octet-stream" http://127.0.0.1:8000/v1/decrypt; done

For sending gRPC requests I used following go code:

conn, err := grpc.Dial("127.0.0.1:8001", grpc.WithInsecure()) if err != nil { t.Fatal(err) } defer conn.Close() client := grpc1.NewReaderClient(conn) for start := time.Now(); time.Since(start) < time.Second * 10; { _, _ = client.Decrypt(context.Background(), &grpc1.DecryptRequest{ ClientId: []byte("testclientid"), ZoneId: []byte("test_zone"), Acrastruct: []byte("blabla"), }, grpc.EmptyCallOption{}) }

While those applications were running, I sent (multiply times) SIGHUP signal to the AcraTranslator and observed that service restarted and there were no cut connections. Testing without connections are rather straightforward.
enhancement refactoring acra-translator
opened by storojs72 8
Questions on Acra, python and TLS

Hi,

I am having problems with latest release of Acra and I don't understand how to troubleshoot that problem. Print is working (empty) but I cannot submit the data into the database with python examples. I am using Ubuntu 20.04.4 LTS.

Error output: (I cannot bold/code style it so I will make "----" where code starts and where ends)

ubuntu@Gallardo:~/acra$ python3 examples/python/example_without_zone.py --public_key=docker/.acrakeys/acra-server/1ec6f16c36e6a3f1a064d708cba5b1ca760ba5ae29d3d80aab67b043ca641255c21ddf8f0fb09a8de336d555b4b961953200c7125f745cff7ed577f7c4b555c4_storage.pub --db_user=${POSTGRES_USER} --db_password=${POSTGRES_PASSWORD} --host=127.0.0.1 --port=5432 --data="data1" insert data: data1 Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1719, in _execute_context context = constructor( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1091, in _init_compiled param = { File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1092, in key: processorskey File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/sql/type_api.py", line 1631, in process return impl_processor(process_param(value, dialect)) File "/home/ubuntu/.local/lib/python3.8/site-packages/acrawriter/sqlalchemy/init.py", line 31, in process_bind_param return create_acrastruct(value, self._public_key) File "/home/ubuntu/.local/lib/python3.8/site-packages/acrawriter/init.py", line 40, in create_acrastruct random_kp = GenerateKeyPair(KEY_PAIR_TYPE.EC) File "/home/ubuntu/.local/lib/python3.8/site-packages/pythemis/skeygen.py", line 42, in init if themis.themis_gen_ec_key_pair( File "/usr/lib/python3.8/ctypes/init.py", line 386, in getattr func = self.getitem(name) File "/usr/lib/python3.8/ctypes/init.py", line 391, in getitem func = self._FuncPtr((name_or_ordinal, self)) AttributeError: python3: undefined symbol: themis_gen_ec_key_pair

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "examples/python/example_without_zone.py", line 73, in write_data(args.data, connection) File "examples/python/example_without_zone.py", line 33, in write_data connection.execute( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1306, in execute return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection return connection._execute_clauseelement( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement ret = self._execute_context( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1725, in _execute_context self.handle_dbapi_exception( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2043, in handle_dbapi_exception util.raise( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise raise exception File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1719, in _execute_context context = constructor( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1091, in _init_compiled param = { File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1092, in key: processorskey File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/sql/type_api.py", line 1631, in process return impl_processor(process_param(value, dialect)) File "/home/ubuntu/.local/lib/python3.8/site-packages/acrawriter/sqlalchemy/init.py", line 31, in process_bind_param return create_acrastruct(value, self._public_key) File "/home/ubuntu/.local/lib/python3.8/site-packages/acrawriter/init.py", line 40, in create_acrastruct random_kp = GenerateKeyPair(KEY_PAIR_TYPE.EC) File "/home/ubuntu/.local/lib/python3.8/site-packages/pythemis/skeygen.py", line 42, in init if themis.themis_gen_ec_key_pair( File "/usr/lib/python3.8/ctypes/init.py", line 386, in getattr func = self.getitem(name) File "/usr/lib/python3.8/ctypes/init.py", line 391, in getitem func = self._FuncPtr((name_or_ordinal, self)) sqlalchemy.exc.StatementError: (builtins.AttributeError) python3: undefined symbol: themis_gen_ec_key_pair [SQL: INSERT INTO test_example_without_zone (data, raw_data) VALUES (%(data)s, %(raw_data)s) RETURNING test_example_without_zone.id] [parameters: [{'data': b'data1', 'raw_data': 'data1'}]]

File used with command source to set environment variables both in Docker compose starting window and the example one:

export ACRA_SERVER_MASTER_KEY=$(echo -n "6EaxBnRwcNZnNyExInkrY0hC6Ahxz5pS" | base64) export ACRA_CONNECTOR_MASTER_KEY=$(echo -n "oPZktQOFIHuyOc65fnoBnh8wWFdixJly" | base64) export ACRA_CLIENT_ID="Testbed" export POSTGRES_DB="test" export POSTGRES_USER="test" export POSTGRES_PASSWORD="test" export MYSQL_ONETIME_PASSWORD="test" export MYSQL_ROOT_PASSWORD="test" export MYSQL_DATABASE="test" export MYSQL_USER="test" export MYSQL_PASSWORD="test" export ACRA_HTTPAUTH_USER=test export ACRA_HTTPAUTH_PASSWORD=test

I hope someone can help me to carry out this problem, Cheers.
question

opened by soluzioninformatiche 7
Error with a golang dependency when using Go 1.10
Hello, I'm trying to build Acra from sources but I'm failing at the first step of the guide which is to install acra-keymaker.

This is the error I get when I run go get github.com/cossacklabs/acra/cmd/acra-keymaker

ubuntu@ip-10-1-1-1:~/work/src/go.opencensus.io$ go get github.com/cossacklabs/acra/cmd/acra-keymaker package go.opencensus.io/exporter/jaeger: cannot find package "go.opencensus.io/exporter/jaeger" in any of: /usr/lib/go-1.10/src/go.opencensus.io/exporter/jaeger (from $GOROOT) /home/ubuntu/work/src/go.opencensus.io/exporter/jaeger (from $GOPATH)

This got resolved when I downgraded go.opencensus.io/ from latest to v0.19.x because the package jaeger is moved to contrib.go.opencensus.io
question docs
opened by Anirudh-MV 7
Don't escape space characters in strings
Found that valid strings with \n characters for string types we encode as hex values and then it rendered incorrectly in web apps. In this PR update our check of printable characters that skipped for encoding. Also found that postgresql sends unicode values in another way then simple latin strings (with encoding into hex). But I didn't find how to fix it correctly, so I created T2531 for futher investigations. And this PR just short fix that should fix obvious and frequent cases.

Checklist

[X] Change is covered by automated tests

[X] The coding guidelines are followed

~~[ ] Public API has proper documentation in the Acra documentation site or has PR on documentation repository with new changes~~

~~[ ] CHANGELOG.md is updated (in case of notable or breaking changes)~~

[X] CHANGELOG_DEV.md is updated

~~[ ] Benchmark results are attached (if applicable)~~

~~[ ] Example projects and code samples are up-to-date (in case of API changes)~~
opened by Lagovas 6
Separate TLS config for AcraConnector and DB

Currently AcraServer uses only one TLS configuration for everything. This means, in particular, that both AcraConnector and DB are expected to use the same private CA, if root certificates are not available in the system certificate store. This is not always the case, and it would be nice to allow separate TLS configurations.

This PR adds several new options:

| New option | Old option | Purpose | | -- | -- | -- | | tls_client_ca | tls_ca | path to CAfile with trusted root certificates for AcraConnector verification | | tls_client_cert | tls_cert | path to server certificate presented to AcraConnectors | | tls_client_key | tls_key | path to private key to the certificate above | | tls_client_auth | tls_auth | TLS authentication level when accepting AcraConnector connections |

| New option | Old option | Purpose | | -- | -- | -- | | tls_database_ca | tls_ca | path to CAfile with trusted root certificates for database verification | | tls_database_cert | tls_cert | path to client certificate used when connecting to database | | tls_database_key | tls_key | path to private key to the certificate above | | tls_database_auth | tls_auth | TLS authentication level when connecting to database | | tls_database_sni | tls_db_sni | server name expected when connecting to database (renamed option) |

New options override old options. The old options still remains not deprecated, they might be useful as a shortcut.

This required a refactoring to split the TLS configuration, ~~so in the future it will be possible to have more specific configuration (think, different allowed authentication modes, cryptosuites, TLS versions, etc.)~~ which allowed to easily split the configuration.

There are no integration tests for this option currently. (Do we need them?) I have tested it manually, with recently updated certificates. The timing could not have been better...
enhancement acra-server

opened by ilammy 6
Split integration tests
The idea of that PR is to split test.py file into pieces as now it's difficult to work with that file. IDEA needs more time to index the file, plus its more difficult to find the class/mixing you need.

What have been made in this PR:

move all encryptor config files from tests/* to tests/encryptor_configs folder.

split test.py into several subfiles with tests:

base.py contains all general classes/constants/mixings etc

searchable_transparent_encryption.py contains base mixing for searchable/transparent encryption tests and their related tests;

tokenization.py - contains base mixing for tokenization/masking tests and their related tests;

type_aware.py - contains all tests related to TA

integrations.py - contains all tests with integrations (AWS, Vault, Consul, etc)

test.py has less than 5k of codes now and contains general tests unrelated to the previously described. The idea was to save test.py file as the main file, not to change the CI running scripts.

Currently, PR is in a draft state, as some test files can be added and structure can be changed.

Checklist

[ ] Change is covered by automated tests

[ ] The coding guidelines are followed

[ ] Public API has proper documentation in the Acra documentation site or has PR on documentation repository with new changes

[ ] CHANGELOG.md is updated (in case of notable or breaking changes)

[ ] CHANGELOG_DEV.md is updated

[ ] Benchmark results are attached (if applicable)

[ ] Example projects and code samples are up-to-date (in case of API changes)
opened by Zhaars 0
[ISSUE] tls_ocsp_from_cert: ignore doesn't ignore database OCSP, undocumented behaviour
Describe the bug

A clear and concise description of what the bug is.

The configuration value tls_ocsp_from_cert: ignore is not working correctly with the database connection. To get the expected behaviour, I had to use an undocumented setting tls_ocsp_database_from_cert: ignore which I guessed from reading the code, not the docs.

My expectation would be for it to be documented here: https://docs.cossacklabs.com/acra/configuring-maintaining/tls/ocsp/

I'm aware of the implications of switching off all these TLS settings and using the same certificate & key for client & server - honestly, I am just trying to get it to work with Python and the asyncpg library, with Cockroach Labs serverless DB which seems to not have OCSP set up as Acra expects it to be (separate bug? I don't know yet.)

To Reproduce My YAML configuration (db_host is censored):

# acra.yml - both encryptor and server config in one file because they don't seem to step on each other. version: 0.94.0 schemas: - table: tbl_auditlog columns: - dc_entry_id - dc_guild - stamp encrypted: - column: dc_user searchable: true - column: dc_target searchable: true - column: py_auditlogentry - table: t_bot_settings columns: - ky encrypted: - column: dc_user searchable: true - column: json_val postgresql_enable: true db_host: "example.cluster.cockroachlabs.cloud" db_port: 26257 keystore_cache_size: -1 keystore_cache_on_start_enable: false tls_key: "./keys/opensslKey.key" tls_cert: "./keys/opensslCertificate.crt" tls_auth: 0 tls_client_id_from_cert: false tls_ocsp_required: allowUnknown tls_ocsp_from_cert: ignore #tls_ocsp_database_from_cert: ignore tls_crl_from_cert: ignore tls_crl_database_from_cert: ignore #tls_ocsp_url: "" #tls_ocsp_client_url: "" #tls_ocsp_database_url: "http://r3.o.lencr.org/ocsp"

My acra-server command: acra-server --config_file=./acra.yml --client_id=dev_acra_client --encryptor_config_storage_type=filesystem --encryptor_config_file=./acra.yml -v -d

My test client (Python):

import asyncio import asyncpg import ssl sslctx1 = ssl.create_default_context( ssl.Purpose.SERVER_AUTH, cafile="./keys/opensslCertificate.crt" ) sslctx1.load_cert_chain( "./keys/opensslCertificate.crt", keyfile="./keys/opensslKey.crt" ) sslctx1.check_hostname = False sslctx1.verify_mode = ssl.CERT_NONE async def maintest(q): c = await asyncpg.connect( host="0.0.0.0", port=9393, user='user', password='qvr Trqnaxra fvaq serv', database="dev", ssl=sslctx1 ) x = await c.fetch(q) print(x) return x asyncio.run(maintest("select * from pg_catalog.pg_user;"))

Expected behavior

A clear and concise description of what you expected to happen.

Acra should ignore OCSP URLs on the certificates, on both connections. The problem behaviour only happens when commenting out the tls_ocsp_database_from_cert: ignore line in the acra.yml file and then restarting the Acra server. The expected behaviour happens when it is uncommented.

Acra configuration files

For AcraServer:

[x] configuration file or CLI params that you use to start AcraServer;

[x] encryptor_config.yaml if used.

Environment (please complete the following information):

Acra version: [0.94.0+bullseye on Debian installed with the instructions in the docs]

Database server and its version: [Cockroach DB Serverless, whatever the current version is (Postgres based)]

Installed components:

[x] AcraServer

[ ] AcraTranslator

Data-in-transit encryption between Acra and the client-side application:

[x] TLS

[ ] AcraConnector

[ ] no transport encryption

Installation way:

[ ] via Docker

[x] via package manager

Additional context

Add any other context about the problem here.

Failure debug logs will be attached.
bug question docs
opened by Burrito5152 2
Add consistent tokenization as default config option
Added consistent tokenization to default option of encryptor config

Checklist

[ ] Change is covered by automated tests

[ ] The coding guidelines are followed

[ ] Public API has proper documentation in the Acra documentation site or has PR on documentation repository with new changes

[ ] CHANGELOG.md is updated (in case of notable or breaking changes)

[ ] CHANGELOG_DEV.md is updated

[ ] Benchmark results are attached (if applicable)

[ ] Example projects and code samples are up-to-date (in case of API changes)
opened by Zhaars 0
Keep in mind utf8 strings in prefixed hashes
Here is merged master into your branch plus fixes with calculating prefixes from utf8 strings instead of encoded arrays. You can review last 2 commits after merge branch or pull changes from master into your branch and refresh.

Checklist

[ ] Change is covered by automated tests

[ ] The coding guidelines are followed

[ ] Public API has proper documentation in the Acra documentation site or has PR on documentation repository with new changes

[ ] CHANGELOG.md is updated (in case of notable or breaking changes)

[ ] CHANGELOG_DEV.md is updated

[ ] Benchmark results are attached (if applicable)

[ ] Example projects and code samples are up-to-date (in case of API changes)
opened by Lagovas 0

Releases(0.94.0)

0.94.0(Nov 16, 2022)
0.94.0, November 17th 2022

Core:

In this release we deprecated Zones functionality and all flags and CLI parameters related to it. These flags will be removed in the next versions. Acra will warn about deprecations.

Acra Community Edition supports separate encryption keys linked to the ClientIDs and allows to manage key switching via TLS certificates.

Acra Enterprise Edition supports more flexible mapping between users/apps and encryption keys via SQL variables.

AcraServer, AcraTranslator, AcraKeymaker, AcraKeys, AcraRotate, AcraAddZone, AcraBackup, AcraLogVerifier, AcraPoisonRecordMaker, AcraRollback:

Added new CLI flags for better KMS support (documentation page, #552, #553, #554):

--kms_credentials_path=<filepath> - path to configuration file specific for KMS type

--kms_type=[aws] - type of KMS provider

Added support of encrypting the Acra Master Key using AWS KMS key (key wrapping technique). AWS KMS documentation page, #552.

Added support of the several encryption strategies for keys in the keystore (#556) and added new CLI flag:

--keystore_encryption_type - specifies type of keys encryption for keystore. Accepts env_master_key, vault_master_key, kms_encrypted_master_key, kms_per_client. Read description of types on documentation pages of appropriate tools, for example AcraKeymaker.

Extended configuration of TLS options when storing ACRA_MASTER_KEY in HashiCorp Vault. #578

Added 12 flags related to OCSP/CRL support. You can find all of these flags in documentation on pages related to appropriate tool, for example AcraKeymaker.

AcraServer, AcraTranslator, AcraKeymaker, AcraKeys, AcraRotate, AcraAddZone, AcraTokens:

Added TLS support for Redis storage for Keystore. Added new 15 CLI flags related to TLS configuration. Read more on appropriate tool's page, for example AcraKeymaker. #566, #565

AcraServer, AcraTranslator, AcraKeymaker, AcraKeys, AcraRotate, AcraAddZone, AcraRollback:

Deprecated all Zones related CLI flags and API descriptions #577

AcraServer, AcraTranslator:

Improved resistance against memory leakage: in-memory cache for keystore now uses randomly generated symmetric key for key encryption instead of ACRA_MASTER_KEY. #555

Improved reloading on SIGHUP signals. #557

AcraServer:

Added support of HashiCorp Consul as a configuration source for encryptor config. Acra can load configuration from the Consul instead of file. Added new CLI flag (--encryptor_config_storage_type=[filesystem|consul]) to switch source and Consul specific flags. Read more on documentation page about encryptor config and acra-server's configuration description. #568

Improved support of searchable tokenization. AcraServer captures SELECT queries and update WHERE clauses to add support of filtering with consistent tokenization. #581

Improved searchable encryption with more complex queries. #586, #592, #598, #599, #594.

Improved SQL parser (better compatibility across different SQL databases):

Added support of NULLS FIRST/NULLS LAST ordering clauses, joins with subqueries #547

Added support of RETURNING clauses. #584

Improved processing prepared statement. #580, #593

Added new section to encryptor config called database_settings. #532, #590

Contains subsections mysql and postgresql

Currently mysql subsection has one option, case_sensitive_table_identifiers, boolean, to configure whether table names should be considered case-sensitive when comparing with names in encryptor config

Table/column matching now works like this:

For PostgreSQL:

Raw identifiers are case-insensitive

Identifiers wrapped with double quotes are case-sensitive

MySQL:

Column identifiers are always case-insensitive

Table names are case-insensitive by default, could be changed with case_sensitive_table_identifiers option mentioned above Case-insensitive means the identifier is converted to lowercase before comparing with values from encryptor config, encryptor config should contain lowercase version of column/table name. Case-sensitive means identifiers are compared with values from encryptor config "as is", encryptor config should contain exactly the same identifier as in database schema.

Removed deprecated --tls_db_sni flag. Now only --tls_database_sni is available. #564

Added support of separate configuration and specifying of CRL/OCSP settings for connections from database and applications. Added flags: --tls_ocsp_[database|client]_required, --tls_[ocsp|crl]_[database|client]_check_only_leaf_certificate, --tls_[ocsp|crl]_[database|client]_from_cert, --tls_[ocsp|crl]_[database|client}_cache_size, --tls_[ocsp|crl]_[database|client}_cache_time, --tls_[ocsp|crl]_[database|client}_cache_size. You can find all of these flags in documentation on pages related to appropriate tool, for example AcraServer. #564.

AcraTranslator:

Improved HTTP API performance. Refactored HTTP processing core. Now AcraTranslator uses golang's HTTP server with gin router #550. Added support of:

HTTP 2.0 connections additionally to HTTP 1.1

Keep alive connections

Added TLS support for HTTP API:

--http_api_tls_transport_enable=[true|false] new flag added to turn on accepting TLS connections instead of raw TCP. Works only together with --http_api_enable=true. #550

Example projects and demos:

Python examples: updated to show searchable encryption feature. #548

Source code(tar.gz)
Source code(zip)
0.93.0(May 27, 2022)
0.93.0, May 27th 2022

This release brings type awareness which improves transparent encryption on AcraServer. Type awareness means that it's possible to tell AcraServer what are the original data types for fields. During decryption, AcraServer will convert decrypted fields to their original data types. No need to change client application code to work with "binary data".

It's also possible to choose a default value for each data field if its decryption failed. AcraServer can send a a default value like "<encrypted data>" instead of decryption errors, making developers' and users' life easier.

Core:

AcraServer:

Added type awareness and ability to map binary data to a certain data type when sending decrypted data back to the application. Extended encryptor_config which allow configure mapping application data type to proper database's type. #515, #517, #523, #519, #520

Extended encryptor_config with new parameters:

data_type - specify data type expected by application. Accept str, bytes, int64, int32 values. #515, #517

default_data_value - specify a placeholder (default value) to replace data that couldn't be decrypted. #515, #517

response_on_fail - specify action on decryption failure. Accepts ciphertext (returns encrypted data as is), default_value (returns values from default_data_value parameter), error (returns error as DB error with message like encoding error in column {column_name}). #521, #533

Deprecated tokenize parameter in encryptor_config and changed focus on token_type parameter. Now is enough to specify token_type parameter without tokenize: true to turn on tokenization. Read more in the documentation. #527

Removed auto-generation poison record's keys but leaved for acra-poisonrecordmaker. It improves decryption due to omitting extra key generation and poison record recognition. #516

Improvements in handling error cases on DB protocol layer. #511, #515, #517, #520, #528, #535, #537

Improved sql parser and support of set command. #534

Ignored legacy keys on startup loading to cache. #510, #522

Improved PostgreSQL/MySQL protocol support. #525, #526, #539, #540, #541, #542, #543, #544

AcraCensor:

Removed legacy IsForbidden field from acra-censor’s logs. Read more here in notes. #508

AcraKeys:

Removed duplicate entries in list command. #530

Other:

Makefile target install_dev_deps install required golang's dependencies for development and code generation. #531

Documentation:

Improved description of AcraServer's encryptor_config, adding details and examples about data processing options: encryption, searchable encryption, masking, tokenization, type awareness, etc.

Updated "Debugging and troubleshooting" section with more tips and tricks.

Example projects and demos:

Python examples: updated to show type masking feature. #524, #529

acra-engineering-demo updated to show data type masking support. #46, #47, #48, #49, #50, #51.

Source code(tar.gz)
Source code(zip)
0.92.0(Mar 1, 2022)
0.92.0, March 1st 2022

This release brings stability and performance fixes to AcraServer and AcraTranslator. It officially deprecates usage of AcraConnector in favour of TLS everywhere. Some default configuration params are changed in favour of more secure & better performance settings.

Core:

AcraServer, AcraTranslator:

Improved TLS certificate validation performance with CRL. #482

Poison record detection turned off by default. Flag --poison_detect_enable changed default value from true to false. #484

Removed SecureSession and AcraConnector support as transport encryption. #481

Improved and clarified log messages. Removed messages with error level for success cases (not detected poison record), clarified context of messages. #487

Added suggestions in log messages how to solve issues with TLS connections. #493

Improved in-memory caching keys:

Added caching symmetric keys like asymmetric #489

Added caching metadata about rotated keys #498

Added new flag --keystore_cache_on_start_enable that turns on loading all keys into in-memory cache on startup. #497

Changed default value for --keystore_cache_size parameter from -1 (which means no limits for cache) to 1000 (cache items). #497

AcraServer:

The default CryptoEnvelope has changed from acrastruct to acrablock in the encryptor_config. Now AcraServer will use faster encryption by default. You can select which CryptoEnvelope to use in encryptor_config. See AcraStructs vs AcraBlocks documentation, #485

Extended PostgreSQL's SQL syntax support with null::<type> type casts. #479

Changed the default values for next CLI parameters:

--tls_client_id_from_cert changed from false to true. Now AcraServer require app's TLS certificates and map them to keys. #481

Improved performance for:

querycapture handler in AcraCensor. #483

transparent encryption and poison record detection. #487, #496

searchable encryption. #490

Removed next CLI parameters due to removed AcraConnector support:

--securesession_id, --acraconnector_tls_transport_enable, --acraconnector_transport_encryption_disable. #481

AcraTranslator: Now AcraTranslator works with TLS by default, it doesn't support AcraConnector anymore. #481

Removed next CLI parameters due to removed AcraConnector support:

--securesession_id, --acratranslator_tls_transport_enable, --acraconnector_transport_encryption_disable. #481

AcraKeymaker:

Now handle correctly generation symmetric keys into not existing folders. #486

Removed next CLI parameters due to removed AcraConnector support:

--generate_acraconnector_keys, --generate_acraserver_keys, --generate_acratranslator_keys. #481

AcraKeys:

Removed next key types for all commands (generate, read, destroy): transport-connector, transport-server, transport-translator. #481

AcraConnector:

Removed everywhere and stopped support. Switch to TLS instead, see Security controls > Transport Security > TLS, #481

Example projects and demos:

Python examples: now support MySQL database. #476

Source code(tar.gz)
Source code(zip)
0.91.0(Dec 16, 2021)
0.91.0, December 16th 2021

Core:

AcraWebConfig, AcraAuthManager:

Have been deprecated and not supported anymore #456. The following changes have been made:

removed docker/acra-authmanager.dockerfile, docker/acra-webconfig.dockerfile files.

removed acra-webconfig and acra-authmanager from docker/acra-build.dockerfile file that is base image for all other service's images.

removed acra-webconfig and acra-authmanager from all docker/docker-compose.*.yml files.

reserved event codes for log entries in range [550, 558] related to AcraWebConfig.

AcraConnector:

Has been deprecated and will be removed in the next releases. AcraServer and AcraTranslator will accept only direct TLS. See Transport security/TLS. connections from applications.

Removed mentions and usage from acra-engineering-demos.

AcraKeymaker:

Some keys can be configured without ClientID #454.

Removed --generate_acrawebconfig_keys flag according to AcraWebConfig/AcraAuthManager deprecation #456.

AcraKeys:

read command supports symmetric encryption keys with ClientID and ZoneID #472.

generate command:

supports rotation for symmetric encryption keys with ZoneID #472.

deprecates next flags: --acraconnector_transport_key, --acraserver_transport_key, --acratranslator_transport_key.

removed --acrawebconfig_symmetric_key flag according to AcraWebConfig/AcraAuthManager deprecation #456.

Improved handling CLI parameters related to Redis #459.

AcraServer:

Removed --auth_keys parameter according to AcraWebConfig/AcraAuthManager deprecation #456.

Removed /loadAuthData, /getConfig, /setConfig endpoints from HTTP API according to AcraWebConfig/AcraAuthManager deprecation #456.

AcraTranslator:

Accepts POST HTTP request method additionally to GET for v2 API. Method GET marked as deprecated and warns with log message. Deprecated HTTP GET method was used. Please use HTTP POST method instead. if was used #466.

Infrastructure:

Build binaries with -tags netgo flag, that forces usage of Go resolver to solve issues related to resolving hostnames. between Docker containers. Updated acra-build.dockerfile used as base image for all cossacklabs/acra-* images (#452).

Added missing parameter --keystore=v1 for existing docker-compose files that caused errors (#452).

Documentation:

Has been updated :)

Improved guide about integration AcraTranslator into infrastructure.

Extended description for AcraTranslator's HTTP API.

Example projects and demos:

Python examples: now work with TLS connections to AcraServer/Database. Also has been updated sqlalchemy version and binary column type from Binary to LargeBinary #463.

acra-engineering-demos don't illustrate AcraConnector usage anymore. All applications and services connect to AcraServer directly.

Source code(tar.gz)
Source code(zip)
0.90.0(Nov 9, 2021)
0.90.0, November 09th 2021

New

Updated documentation

Acra's documentation is now open-source and updated for this release. Please find use cases, usage scenarios, data flows, descriptions of security controls, cryptography deep dive, scaling and load balancing, optimisations and many more.

Check out the updated documentation.

Searchable encryption

Two components can provide searchable encryption functionality:

AcraServer — transparent searchable encryption of fields marked as searchable in encryptor_config for INSERT and UPDATE queries, calculating hash and searching by hash for SELECT queries, with per column configuration.

AcraTranslator — provides gRPC and HTTP API calls to encrypt data field into searchable form, and to generate searchable hash from the plaintext search query.

Read more details in the Acra documentation section dedicated to Searchable encryption.

Masking

AcraServer – provides masking functionality. It is transparent masking for INSERT and UPDATE queries, and transparent demasking for SELECT queries, with per column configuration. Read more details in the Acra documentation section dedicated to Masking.

Tokenization (Pseudonymisation)

Two components can provide tokenization functionality:

AcraServer — transparent tokenization for INSERT and UPDATE queries, and transparent detokenization for SELECT queries, with per column configuration.

AcraTranslator — provides gRPC and HTTP API to tokenize or detokenize the field.

Read more details in the Acra documentation section dedicated to Tokenization.

AcraBlock

AcraBlock is a symmetric cryptographic container and is faster and more compact than AcraStruct. It used on AcraServer side in transparent encryption, masking, tokenization, searchable encryption.

AcraTranslator supports AcraBlocks in encryption, searchable encryption and tokenization via gRPC and HTTP API.

Read more details in the Acra documentation section dedicated to AcraBlock.

KeyStore v2

Added new storage format for keys in KeyStore that cryptographically strong key integrity checks, additional tracking metadata simplifying key management, KMS integrations.

Read more details in the Acra documentation about difference between two versions.

HashiCorp Vault integration

All Acra services that work with encryption/intermediate keys can load master key ACRA_MASTER_KEY from HashiCorp Vault. Previously was supported only environment variables.

Read more details on our KMS integration page in the documentation.

Core

AcraServer

We recommend using AcraServer in transparent encryption mode, connecting to it via TLS from application side. Use AcraServer with AcraBlocks for faster & more efficient configuration. AcraConnector and AcraWriter are optional components, and can be omitted.

Read more details on our Integrating AcraServer into infrastructure.

Added prepared statements support for MySQL. Now all transparent operations over the data works with prepared statements too.

Extended and refactored TLS related CLI parameters.

tls_client_id_from_cert - switching to new mode with clientID extraction from certificates instead of handshakes with AcraConnector or static mode with --client_id parameter.

OCSP-related:

tls_ocsp_url, tls_ocsp_client_url, tls_ocsp_database_url - URL of OCSP server to use, for acra-server may be configured separately for both directions.

tls_ocsp_required - whether to allow "unknown" responses, whether to query all known OCSP servers (including those from certificate).

tls_ocsp_from_cert - how to treat URL listed in certificate (use or ignore, whether to prioritize over configured URL).

tls_ocsp_check_only_leaf_certificate - whether to stop validation after checking first certificate in chain (the one used for TLS handshake).

CRL-related:

tls_crl_url, tls_crl_client_url, tls_crl_database_url - URL of CRL distribution point to use, for acra-server may be configured separately for both directions.

tls_crl_from_cert - how to treat URL listed in certificate (use or ignore, whether to prioritize over configured URL).

tls_crl_check_only_leaf_certificate - whether to stop validation after checking first certificate in chain (the one used for TLS handshake).

tls_crl_cache_size - how many CRLs to cache in memory.

tls_crl_cache_time - how long cached CRL is considered valid and won't be re-fetched.

Separated parameters for connections accepted from application/AcraConnector or established to database with TLS:

acra-server's certificate: tls_client_cert and tls_database_cert (overrides tls_cert).

acra-server's key: tls_client_key and tls_database_key (overrides tls_key).

CA certificate path: tls_client_ca and tls_database_ca (overrides tls_ca).

TLS authentication: tls_client_auth and tls_database_auth (overrides tls_auth).

Supports RETURNING syntax in SQL queries with proper decryption data in the response.

--sql_parse_on_error_exit_enable new flag that force acra-server to stop query execution if can't parse SQL query. By default, it is false.

Improved encryptor config validation.

Deprecated --acrastruct_wholecell_enable and --acrastruct_injectedcell_enable flags and will be ignored. Now acra-server works as in InjectedCell mode.

Deprecated --tls_db_sni parameter and replaced with tls_database_sni.

AcraTranslator

We recommend using AcraTranslator as gRPC or HTTP API, connecting to it via TLS from application side. Use AcraTranslator with AcraBlocks for faster & more efficient configuration.

Read more details on our Integrating AcraTranslator into infrastructure.

--acratranslator_client_id_from_connection_enable flag turns on mapping TLS certificates to encryption keys with .

Extended HTTP API as version 2 and gRPC API with supporting all new features like Searchable encryption, Tokenization, symmetric key encryption with AcraBLock and synchronized with gRPC API.

HTTP API version 2 with OpenAPI and Swagger support.

AcraServer, AcraTranslator

audit_log_enable - new parameter turns on cryptographically signed audit logging. Read more in the Acra documentation.

Support direct TLS connections from applications without AcraConnector. acra-server and acra-translator will map client's certificates to proper encryption keys in KeyStore.

tls_identifier_extractor_type - new parameter that configures strategy of extraction metadata from certificates for mapping to clientID (default: distinguished_name, another option: serial_number).

AcraServer, AcraTranslator, AcraConnector

TLS certificate validation using OCSP and CRL. All services and tools that accepts incoming connections can be configured with new rules of connection validation. Read more details in the Acra documentation section dedicated to TLS configuration.

--log_to_console - parameter turns on\off logging to stderr.

--log_to_file - parameter specify path to file for logs. May be used together with logging to stderr.

AcraKeymaker

New flags to generate new kind of keys for new features:

--generate_hmac_key - flag turns on generation symmetric key for HMAC used in searchable encryption.

--generate_log_key - flag turns on generation symmetric key for cryptographically signed audit logging.

--generate_symmetric_storage_key - flag turns on generation symmetric key for encryption with AcraBlocks.

--keystore - specify version of KeyStore. Now supported v1 (default) and v2 (new) versions.

New flags to generate encryption keys for TLS certificates:

--tls_cert - specify client's TLS certificate to generate encryption keys. Should be used instead --client_id flag.

--tls_identifier_extractor_type - switch type of ClientID extraction from TLS certificate. Supports distinguished_name (default) and serial_number values.

AcraAddZone

--fs_keystore_enable now is deprecated and ignored.

AcraTokens

acra-tokens is a new command-line utility used for managing generated tokens with turned on tokenization. Tokens may be stored in BoltDB or Redis for now. Read more details in the Acra documentation.

AcraBackup

acra-backup is a command-line utility used for storing and managing the keystore backups. Also, it helps to migrate keys from one KeyStore to another one by export + import operations.

Read more details in the Acra documentation on acra-backup page.

AcraKeys

acra-keys is a command-line utility used for different keys operations especially for v2 keystore. It consists of several subcommands each of which is responsible for a separate functionality.

Other

Support of RHEL >= 7

Build image use Debian 10 instead of Debian 9

Configurable build and install parameters in Makefile (see make help)

Self-documented Makefile

Makefile pkg target with automatic detection of OS (use it instead of rpm and deb)

Makefile targets dist, temp_copy

docker_push target replaced with docker-push

Source code(tar.gz)
Source code(zip)
0.85.0(Mar 15, 2019)
0.85.0, March 15th 2019

Core:

Breaking changes:

Introducing a new more flexible configuration format for AcraCensor rules. AcraCensor doesn't support the old format, all users should migrate (don't worry, it's a simple procedure).

Search through encrypted data

You now can run SQL queries over encrypted AcraStructs allowing users to search through sensitive data without exposing it. This feature is only available in Acra Enterprise version.

Transparent proxy mode

TLDR: Transparent proxy mode allows you to configure AcraServer to encrypt records in specific database columns without altering the application code.

The application flow doesn't need to change: application sends SQL requests through AcraConnector and AcraServer to the database. AcraServer parses each request, encrypts the desired values into AcraStructs, and passes the modified requests to the database. To retrieve the decrypted data, your application talks to AcraServer again: upon receiving the database response, AcraServer tries to detect AcraStructs, decrypts them, and returns the decrypted data to the application.

Transparent proxy mode is useful for large distributed applications where updating the source code of each client app separately would be complicated.

To enable this mode, you need to create a separate encryptor configuration file (acra-encryptor.yaml) that describes which columns to encrypt and provide a path to it in the AcraServer configuration file (or via CLI params --encryptor_config_file=acra-encryptor.yaml).

Read more details in the Readme and in the Acra documentation section dedicated to Transparent encryption.

(#285, #309, #314).

AcraCensor – SQL firewall to prevent SQL injections

TLDR: Improved stability of AcraCensor, switched to more flexible rules' configuration.

Breaking changes: Introducing a new format for configuration files, the previous format is no longer supported, you should migrate to the new one.

New configuration file format allows configuring the allowlist and the denylist separately or simultaneously.

The allow handler allows something specific and restricts/forbids everything else. The allowall handler should be a final statement as that means that all the other queries will be allowed.

The deny handler allows everything and forbids something specific. The denyall means "block all queries!" (that haven't been allowed or ignored before).

For each handler, there are settings that regulate queries, tables, and patterns. The order of priority for the lists is defined by their position in the configuration file. The processing priority for each list is as follows: queries, followed by tables, followed by patterns.

(#298, #297, #304, #306).

Read more in AcraCensor docs.

Added version to the configuration file. This allows detecting an outdated configuration easily. From now on, AcraCensor supports explicit configuration version and logs errors if the configuration is not valid (#321).

Improved parsing of SQL queries with prepared statements (#303, #283).

Improved error handling for queries that AcraCensor can't parse (#291, #284).

Added ability to log unparsed queries to a separate log file for the debugging and configuration purposes. Sometimes AcraCensor can't parse all of the incoming queries and it is useful to have a separate log for them.

How to use it: Provide the path to the unparsed queries log file in the configuration file parse_errors_log: unparsed_queries.log (#295).

Improved support of PostgreSQL queries ("RETURNING" clause) and quoted identifiers (now you can use "tablename" and WHERE "column"=1) (#296).

Fixed the bug in QueryCapture log that caused duplicated of records in the log to appear (#318).

AcraServer

Fixed handling of null-size packets in PostgreSQL protocol (#286).

Fixed handling of setting a custom connection API port (#294).

Fixed handling of the plain text data response: if the database returns a plain text response, it is redirected "as is" (#305).

Fixed handling of casted placeholders in expressions like SELECT $1::type1::type2 FROM table1 WHERE column1=$2::type3::type4 (#328).

Improved code quality (some refactoring here and there) (#302, #301).

AcraServer, AcraTranslator, AcraConnector

Refactored logs and error messages got even more descriptive and user-friendly (#312, #299, #317).

Added on-start version logging to make it easier to understand which version is running (#319).

Added versioning for configuration files of each service (#322).

Added exporting version to metrics (#330, #320).

Updated some configuration parameters descriptions for better user-friendliness (please see our docs of AcraConnector and AcraServer for detailed descriptions of each parameter and usage examples) (#329).

AcraWriter

Updated AcraWriter for ActiveRecord (Ruby), fixed dependencies, added support of mysql2 adapter (#287).

Updated AcraWriter for Django (Python), fixed potential encoding issues (#293, #292).

Updated AcraWriter for C++, improved cpp codec usage (#290, #289).

Added bitcode for AcraWriter iOS and added Swift example project (#327, #326, #325, #324, #323, #323, #307).

Improved distribution of AcraWriter for Android, now it's available via Maven (#310).

Other

Added more tests and then — added even more tests. We just love automating things! (#331, #311, #308, #292).

Updated the version of pyyaml used in the tests due to CVE-2017-18342. This change doesn't affect the users of Acra, it only affects our test suite (#300).

Infrastructure:

Updated Docker files, added more comments, and updated Go version (#313, #288).

Example projects and demos:

iOS Swift example project that shows how to generate AcraStructs with and without Zones.

Android example project that shows how to integrate AcraWriter library into Android app using maven, and then to generate AcraStructs with and without Zones, and to decrypt them using AcraTranslator.

AcraCensor demo that shows how to configure AcraCensor for SQL injections prevention in OWASP Mutillidae 2 example app.

Protecting data in a Rails application demo based on AcraServer, PostgreSQL, and Ruby on Rails client application.

Protecting metrics in TimescaleDB demo based on AcraServer, TimescaleDB, and Grafana.

Transparent proxy mode demo that shows how to configure AcraServer in Transparent proxy mode to protect Django-based application.

Related blog posts:

The difference between SQL firewalls and Web Application Firewalls.

Engineering details on how we built AcraCensor.

Features coming soon:

Pseudonymisation: an early version of pseudonymisation library/plugin for Acra for transparent data pseudonymisation.

Cryptographically protected audit log: protection for logs against tampering.

Documentation:

Updated AcraServer documentation to describe Transparent mode in more details.

Updated AcraCensor documentation to describe the new configuration format and procedures for migration from the previous one.

Updated AcraWriter documentation for iOS and Android to reflect the improved installation ways.

Source code(tar.gz)
Source code(zip)
0.84.2(Feb 19, 2019)

0.84.2, February 19th 2019

Hotfix:

Fixed an issue in communication of AcraServer and PostgreSQL that causes AcraServer to stop processing connection due to an unexpected error in parsing packets. The issue occurred when last data in data row column from PostgreSQL comes with empty data (0 bytes).

Details: https://github.com/cossacklabs/acra/pull/315
Source code(tar.gz)
Source code(zip)
0.84.1(Jan 24, 2019)

0.84.1, January 25th 2019

Hotfix:

Fixed an issue in communication of AcraServer with some specific ORMs (xorm precisely) with MySQL database. In some cases, when database has plaintext data, AcraServer can't decrypt it (which is ok), but propagates decryption error and closes connection (which is wrong, it's fixed).

Details: https://github.com/cossacklabs/acra/pull/305
Source code(tar.gz)
Source code(zip)
0.84.0(Nov 9, 2018)
0.84.0, November 9th 2018

Core:

Key management

Improved LRU cache: fixed concurrent access to LRU cache by adding mutex. LRU cache is used for quick access to in-memory keys (private keys are stored encrypted) in AcraServer and AcraTranslator (#272).

AcraServer documentation, AcraTranslator documentation.

Improved AcraRotate utility: added "dry-run" mode for testing AcraRotate before it is used for real. In the "dry-run" mode AcraRotate doesn't rotate keys: it fetches AcraStructs (from files or database), decrypts, rotates in-memory keys, encrypts the data with new public keys and prints the resulting JSON with new public keys without actually saving the rotated keys and AcraStructs. As key rotation might be tricky, we want users to make sure that AcraRotate has all the required permissions and access right before actually re-encrypting the data (#269).

AcraRotate documentation.

AcraWriter

Added C++ AcraWriter library, added examples and tests. The library itself is a single header-only file acrawriter.hpp with dependency on Themis, placed in wrappers/cpp. Read the usage guide and examples in examples/cpp folder (#270).

AcraWriter C++ documentation.

Logging

Improved logs of AcraConnector and AcraServer: use Debug log level for all network errors (closed connection, unavailable network, etc) and use Error log level only for cases of certainly unexpected behavior (#275).

Improved startup logs: log process PID on start of AcraServer, AcraConnector, AcraTranslator, and AcraWebConfig (#275).

Fixed timestamps: do not overwrite logs' timestamps (#273).

Tracing with OpenCensus

Added tracing with OpenCensus: AcraServer, AcraConnector, and AcraTranslator track every request from client application to the database and back. Each client request has a unique traceID that helps measure how much time it needs to perform a certain data processing functions (i.e. checking requests via AcraCensor, encrypting data, decrypting AcraStructs, etc.). Traces can be exported to Jaeger (#279, #276, #274).

You can read more about tracing in our documentation in Tracing in Acra.

A blogpost about technical details, profits, and pitfalls during the implementation of traces is coming soon.

Other

Improved AcraServer's connection handling: stop accepting connections after error and stop AcraServer instead of trying to accept connections after the listening socket was closed (#275).

Improved AcraCensor's handling of prepared statements for PostgreSQL binary protocol (#280).

Improved handling of terminating packets (COM_QUIT for PostgreSQL and TerminatePacket for MySQL) to correctly handle the closing connections from clients (#275).

Refactored inner logic of AcraCensor: improved code quality and stability, added more tests that use more patterns (#268).

Infrastructure:

Ceased testing and supporting Go versions below 1.9. This will only affect the users who build Acra manually from sources. You can install the pre-built Acra components shipped for various Ubuntu, Debian, and CentOS distributives using Installation guide. Alternatively, you can check out our Docker images and Docker-compose files in docker folder (#277).

Tested Acra suite with PostgreSQL v11 and MariaDB v10.3 and updated docker compose examples and Acra Engineering Demo to use it (#278).

Published Acra load balancing demo: it illustrates some of the many possible variants of building high availability and balanced infrastructure based on Acra data protection suite components, PostgreSQL, and Python web application. In these examples we used HAProxy – one of the most popular high availability balancers today.

Updated AcraStruct Validator – an online tool that can decrypt your AcraStructs. AcraStruct Validator is useful for developers who build their own AcraWriters (to validate AcraStruct binary structure).

Features coming soon:

Running SQL queries over encrypted data: perform AcraServer-side lookups (search) over protected data.

Pseudonymisation: early version of pseudonymisation library/plugin for Acra for transparent data pseudonymisation.

Cryptographically protected audit log: protection for logs against tampering.

Documentation:

AcraWriter C++ has a short guide for installing and using AcraWriter for C++.

AcraRotate: added description and notes about "dry-run" mode.

Updated documentation for logging, collecting metrics, and tracing in Acra.

Many small fixes here and there to make your overall experience of using Acra's docs on a new platform distinctive and smooth ;)

Source code(tar.gz)
Source code(zip)
0.83.0(Sep 28, 2018)
0.83.0, September 28th 2018

Core:

Security

Updated the default and allowed TLS configurations (#254).

Use TLS v1.2 (the latest supported version in Golang now) and limited cipher suite recommended by Internet Assigned Numbers Authority and OWASP for most transport connections.

Use TLS v1.0 and extended cipher suit for MySQL TLS connections due to the limited support of MySQL database and drivers.

Improved security of transport connection between Acra's services by validating the clientId length. This decreases the chance of misusing the clientId (#253).

Key management – key rotation

Added AcraRotate utility for rotation of Zone keys and re-encryption of AcraStructs. AcraRotate generates a new Zone keypair (zoneid_zone.pub and zoneid_zone) for a particular ZoneId and re-encrypts the corresponding AcraStructs with new keys. ZoneId stays the same (#256, #239).

AcraRotate doesn't affect the ACRA_MASTER_KEY or storage keypair used without Zones (clientid_storage.pub / clientid_storage keys).

AcraRotate rotates only the Zone storage keys and affects only the AcraStructs encrypted with Zones.

AcraRotate works with AcraStructs stored both in database cells (MySQL or PostgreSQL) or files.

Read the full documentation on AcraRotate on the Documentation Server.

AcraCensor – SQL filter and firewall

Improved SQL filtering through more complex pattern matching (#264, #263, #262, #238).

%%VALUE%% pattern represents literal value (string, binary, number, boolean) and is supported in the following expressions: WHERE, IN, ORDER BY, GROUP BY, BETWEEN.

%%LIST_OF_VALUES%% pattern represents several values one by one, used with IN expressions.

%%SUBQUERY%% pattern represents a subquery expression inside the main query.

%%WHERE%% pattern represents one or more expressions after a WHERE statement. This pattern works for SELECT/UPDATE/DELETE queries.

%%COLUMN%% pattern represents a column expression used after SELECT and ORDER BY expressions.

%%SELECT%% pattern represents a whole SELECT expression.

Read the detailed description and usage examples on the AcraCensor page on DocServer.

AcraWriter

Added Java/Android AcraWriter library, added examples and tests (#252).

Read the usage guide and examples in examples/android_java folder.

Added SQLAlchemy type wrappers for the Python AcraWriter (#257).

Improved and refactored the Python AcraWriter example of encrypting data and reading it from the database (#258).

Prometheus Metrics

Added functionality for exporting the basic metrics of AcraServer, AcraConnector, and AcraTranslator to Prometheus: if incoming_connection_prometheus_metrics_string is set, the service will generate specific metrics (time of connection life, time of processing requests, AcraStruct decryption counters) and push them to Prometheus (#260, #251, #234).

Other

Improved AcraConnector's compatibility with PostgreSQL: AcraConnector now correctly handles the database's denial to use TLS connection (#259).

Added export of CLI parameters for AcraServer, AcraConnector, and AcraTranslator to markdown (#261).

Improved readability of CEF-formatted logs by sorting extension fields in alphabetical order (#255).

Improved quality of our codebase — cleaned up the old unnecessary code (#250).

Infrastructure:

Added AcraRotate as a ready-to-use tool inside AcraTranslator and AcraServer Docker containers (#236).

Documentation:

Made the Documentation Server the primary and the only regularly updated source of documentation for Acra. The most recent version of the documentation, tutorials, and demos for Acra can be found there. The GitHub Wiki documentation for Acra is still available, but is no longer updated starting with the version 0.82.0 (with the exception of Changelog and README files with every new version release).

AcraCensor: updated the details on how the "patterns" filter works.

AcraRotate: added a tutorial for using AcraRotate to rotate Zone keys and re-encrypt the data.

Tons of small fixes here and there to make your overall experience of using Acra's docs on a new platform distinctive and smooth ;).

Source code(tar.gz)
Source code(zip)
0.82.0(Aug 14, 2018)
Core:

AcraTranslator

AcraTranslator is a lightweight server that receives AcraStructs and returns the decrypted data. AcraTranslator doesn’t care about the source of the data, it accepts AcraStructs via HTTP or gRPC API. An application can store AcraStructs anywhere it is convenient: as cells in the database, as files in the file storage (local or cloud storage, like AWS S3). An application sends AcraStructs as binary data and receives plaintext (or decryption error) from AcraTranslator.

However, sending plaintext data over a non-secure channel is a bad idea, so AcraTranslator requires the use of Themis Secure Session encryption channel (which is basically an encrypted TCP/unix sockets). To establish a Secure Session connection, an application doesn’t need to contain the crypto-code itself, only to direct the traffic through AcraConnector instead.

AcraTranslator supports AcraStructs via HTTP and gRPC API, uses in-memory LRU cache to store encryption keys, and detects poison records. AcraTranslator is shipped as a pre-built binary and as a Docker container.

Read the detailed guide on how to install, configure, and run AcraTranslator.

(#213, #212, #207, #205, #204, #203, #200, #199, #198, #197)

Updated AcraConnector to support connections with AcraTranslator (#206).

Logging

Improved startup logs for AcraServer, AcraTranslator, and AcraConnector: now it's easier to understand that the service is up and running (#242).

Added clientId to AcraServer logs: now it's easier to understand which client was sending the request that led to a failed or successful AcraStruct decryption (#214).

Improved logging by masking query parameters: neither AcraServer nor AcraCensor won't leak sensitive query while logging the content (#216, #211).

Poison records

Poison record detection for AcraServer and AcraTranslator can now be turned on and off. Poison records detection is ON by default: AcraServer/AcraTranslator will try to detect poison record and log to stderr if a poison record is detected (#240, #230, #215).

Increased performance of AcraServer/AcraTranslator if poison records are enabled (#232).

Key management

Improved processing of decryption keys for AcraServer, AcraTranslator, and AcraConnector: now it is possible to store private keys encrypted in files and in memory, decrypt them before using, and purge after usage (#202).

Added configurable LRU cache for the in-memory keys: this will increase the performance if you operate with hundreds of storage keys on AcraServer and AcraTranslator (#219).

AcraCensor / SQL filter and firewall

Improved SQL filtering by adding pattern matching: now you can blacklist or whitelist queries that match particular patterns, like SELECT %%COLUMN%% FROM company %%WHERE%%.

Currently supported patterns: %%VALUE%%, %%COLUMN%%, %%WHERE%% and %%SELECT%%.

Read the detailed description and usage examples on the AcraCensor page.

(#248, #247, #246, #245, #243, #238, #231, #226, #217)

Improved AcraCensor performance for queries matching (#208).

AcraWriter

Added iOS/Objective-C AcraWriter library as CocoaPod, added examples and tests.

Read the usage guide and examples in examples/objc folder.

(#241, #235, #233).

Improved security of AcraWriter libs for Go, Ruby, Nodejs, and iOS through zeroing secret keys where it was missing (#244).

AcraRollback

Improved handling of insert query parameter to simplify using AcraRollback from bash (#210).

Other

Improved AcraStruct decryption by multiple validations of AcraStruct format before decrypting. This fix improves AcraServer/AcraTranslator error messages in case of a failed decryption (#201).

Improved stability of integration test suite, trying to avoid 'timed out' errors from CircleCI (#200).

Improved code quality, fixing gofmt and golint issues (#229, #228, #227, #224, #223, #221).

WIP

Adding a way to export decryption metrics (decryption time, number of connections) from AcraServer to Prometeus (#234).

Prototyping AcraRotate tool for rotating the Zone keys easily (#239).

Infrastructure:

Drop testing and supporting go version below 1.8. This will only affect the users who build Acra manually from sources. You can install pre-built Acra components shipped for various Ubuntu, Debian, and CentOS distributives using Installation guide. Alternatively, you can check out our Docker images and Docker-compose files in docker folder (#209).

Added AcraTranslator as pre-built binary and docker container (#222).

Added AcraTranslator and AcraConnector docker-compose files: now it's easy to set up a demo stand just by running one command (#225).

Added AcraRollback and AcraPoisonRecordMaker as ready-to-use tools inside AcraTranslator and AcraServer Docker containers (#236).

Documentation:

Key management: clarified key names and default locations, illustrated public key exchange in details.

AcraServer: improved examples of how to run AcraServer.

AcraTranslator: added description of the service, installation and launching guide, added ready-to-use examples for HTTP and gRPC API.

AcraConnector: added examples of how to run AcraConnector with AcraTranslator.

AcraCensor: added examples of the configuration file, described in details how the "patterns" filter works.

AcraRollback: added examples of running AcraRollback in local mode, which allows storing the decrypted data locally, without pushing it back to the database.

This is the last version of Acra for which the main documentation will be actively updated in the GitHub Wiki. From now on, the most recent version of the documentation, tutorials, and demos for Acra will be available in the official Cossack Labs Documentation Server.

Source code(tar.gz)
Source code(zip)
0.81.0(Jul 6, 2018)
Core:

Prepared Statements

Added support of prepared statements for PostgreSQL/MySQL. Both binary and text response formats are supported (#192).

SQL requests filtering in AcraCensor

AcraCensor got smarter in preventing SQL Injections.

Improved flexibility for parsing queries. If AcraCensor can't parse an SQL query, it is considered as potentially too dangerous to send it to the database, so AcraCensor blocks such "unparseable" queries by default. However, setting the configuration flag ignore_parse_error to true will make AcraCensor ignore the "unparseable" quality of queries and send them to the database anyway. Check out the configuration example in configs/acra-censor.example.yaml (#194).

Added support of complex JOIN queries (#191).

Improved reading/writing QueryCapture log file. Now AcraCensor uses bufferisation before writing queries into a log file. Changed format of QueryCapture log to JSON Line (each query sits in a separate line in a log file instead of having an array of JSON objects) (#193).

Introduced a few fixes here and there, made integration tests for AcraCensor more stable (#184).

Improving MySQL support

We introduced MySQL support just a few Acra releases ago and we continue polishing it. Now we've updated the example projects so you can jump right into the code!

Take a look at how to use Acra for both PostreSQL and MySQL databases in these examples:

Go: see the examples/golang folder (#190).

Ruby: see the examples/ruby folder (#189).

Python: see the examples/python folder (#188).

Other

Updated handling of message formats for PostgreSQL and MySQL protocols (#186).

Improved logging in CEF and JSON formats for high load systems (#195).

Added comprehensive Readme to every project in /examples folder (#196).

Added pre-generated configuration file for AcraAuthmanager. Now it's easier to configure AcraServer using AcraWebconfig (#187).

Documentation:

Updated Acra Architecture and Data flow graphic schemes to better illustrate Acra's components, connections between them, and typical use-cases.

Updated AcraCensor's description to explain how unparseable queries are handled.

Described typical Public Key Infrastructure with some advice on where to put Acra in the general scheme of things.

Described Acra's Security Model, possible threats, and possible consequences of compromisation.

Added a page describing the ways Acra can help you better comply with GDPR.

Source code(tar.gz)
Source code(zip)
0.80.0(May 31, 2018)
0.80.0, May 31st 2018

Core:

Renaming

Global renaming of Acra components and their configuration parameters. We believe that the updated naming will decrease confusion about the components' functions and will make Acra's setup and usage process easier.

Main services:

| Old name | New name | Function | | --- | --- | --- | | AcraServer | AcraServer | decrypts data from the database | | AcraWriter | AcraWriter | encrypts data on the client side | | AcraProxy | AcraConnector | encrypts traffic between the client and the server using Themis Secure Session | | AcraCensor | AcraCensor | firewall, part of AcraServer, blocks suspicious SQL requests to the database | | AcraConfigUI | AcraWebConfig | lightweight HTTP web server for managing AcraServer's certain configuration options |

Utilities:

| Old name | New name | Function | | --- | --- | --- | | acra_rollback | AcraRollback | decrypts the whole database | | acra_genkeys | AcraKeymaker | generates encryption keys for storage and transport of the Acra components | | acra_genauth | AcraAuthmanager | generates user accounts for AcraWebConfig | | acra_genpoisonrecord | AcraPoisonRecordMaker | generates poision records for databases | | acra_addzone | AcraAddzone | generates Zones' header for AcraWriter |

Check the configurations of components inside /configs folder and read Migration Guide for more details (#175, #174, #173, #170, #169, #168).

SSL/TLS

Improved SSL/TLS connections between AcraServer<->AcraConnector and AcraServer<->database. Added TLS authentication mode (tls_auth) argument to the AcraServer/AcraConnector configuration files:

for AcraConnector it indicates how to authenticate AcraServer during a TLS connection;

for AcraServer it indicates how to authenticate database during a TLS connection.

Updated TLS configuration to provide other less strict authentication methods (do not authenticate client from server, ask for any certificate, ask and check) (#171).

SQL requests filtering

Added support of filtering SQL requests for PostgreSQL databases. Now you can setup AcraCensor rules for both MySQL and PostgreSQL databases (#177).

Improved QueryCapture: AcraCensor writes allowed/blocked queries into a separate log file without blocking the main process (#176, #172).

See a detailed description of AcraCensor on the corresponding AcraCensor documentation page.

AcraWriter in Ruby

Updated AcraWriter Ruby wrapper for ActiveRecord tutorial and pushed a new gem (#166).

Key Handling

Added make keys target in the Makefile: one command now generates keys and places them into correct folders for all Acra components (#182, #181).

Improved handling of master key length longer than 32 bytes (#183).

Other

Updated notification when AcraConnector is launched in an environment without netstat (#167).

Updated error handling for AcraServer working with Zones and fix some corner-cases in using PostgreSQL protocol (#186, #179).

Infrastructure:

Even better Docker support

Added more ready-to-use Docker Containers: acra-keymaker, acra-authmanager. As a result, each Acra component is wrapped into a Docker container, allowing you to try Acra into your infrastructures easily.

Added easy-to-use docker-compose files for setting up the whole Acra-based environment connected to MySQL database. Possible configurations include setup with/without SSL, with/without AcraConnector, with/without Zones (#180). Check out the instructions and examples in the /docker folder: we have examples for both MySQL and PostgreSQL databases.

Updated descriptions for official Cossack Labs packages on Docker Hub.

Updated Getting started with Docker guide to make starting out with Acra even easier.

OS

Added support of Ubuntu Xenial, Ubuntu Bionic (added precompiled binaries and tests to make sure that Acra is compiling/building/working well on 16.04/18.04).

Documentation:

Updated tutorials about protecting a Ruby on Rails app and a Django app.

Every single document, code line, and image are updated using the new naming.

Source code(tar.gz)
Source code(zip)
0.77.0(Apr 13, 2018)
Core:

MySQL databases

Added support for MySQL: now you can connect Acra to MySQL databases. Works with any SSL mode: require, allow, disable.

Tested and supported on: MySQL (#155, #140).

Note: Prepared statements are not supported yet, but this feature is coming soon!

Read about the new configurations on the AcraServer documentation page.

Keeping keys in secret

Added encryption for the keys' folder: private keys are now symmetrically encrypted by master_key (#143) for storage.

Added ability to generate public/private keys in the separate folders (#148, #142).

Read more about the current changes in key management here.

Filtering requests for MySQL

Added firewall component named AcraCensor to handle MySQL queries.
You can provide a list of restricted or allowed tables, columns, and exact queries to handle. AcraCensor will pass the allowed queries and return error on forbidden ones. Rules are configured and stored in yaml file. Each request is logged in real time. Moreover, all the queries and their states are logged into a separate log file. (#151, #138, #136, #132, #125, #108).

See a detailed description of AcraCensor on the corresponding AcraCensor documentation page.

Web Config UI

Added lightweight HTTP web server for managing AcraServer's certain configuration options.
You can update the proxy address and port, database address and port, handling of Zone mode and poison records. On saving new configuration, acraserver will gracefully restart and use these settings automatically. The access to thiw web page is restricted using basic auth. (#153, #141, #123, #111).

See the interface screenshot and detailed instructions at Acra Config UI page.

Logging

Added support of new logging formats: plaintext, CEF, and json.
Logging mode and verbosity level is configured for AcraServer, AcraProxy, and AcraConfigUI in the corresponding yaml files. Log messages were slightly improved, custom error codes were added (which we believe will help to understand and debug any issues) (#135, #126, #110).

Read more about the log analysis at Logging page.

Tests

Added many new integartion tests, fixed stability and handling of more complicated use-cases (#150, #147, #137, #117, #116, #115).

Infrastructure:

Docker support

Added Docker Container for every main component: AcraServer, AcraProxy, AcraConfigUI, and key generators (AcraGenKeys and AcraGenAuth). You can find the containers in /docker folder or on the Docker Hub (#139).

Updated Getting started with Docker guide to make starting out with Acra even easier.

Added easy-to-use docker-compose files to launch Acra in different environments, including key distribution. Possible configurations are:

acraserver + acra_configui;

connecting to PostreSQL or MySQL databases;

using Secure Session or SSL as transport encryption;

with or without acraproxy;

with or without zones.

This is huge! We encourage you to try it! Check out the instructions and examples in the /docker folder. (#154, #146, #134, #133, #102).

Go versions

Updated the list of supported versions of Go. Every Acra component can now be built using Go >1.7, except acra_rollback that requires Go >1.8. No worries, you can still download Acra as a binary package anyway :)

OS

Dropped support of Debian Wheezy (no autotests, no precompiled binaries now).

Documentation:

Updated QuickStart documentation about launching and building Acra components.

Described how to setup AcraCensor and AcraConfigUI.

Added more details and described new options (like using TLS and connecting to MySQL databases) for AcraServer and AcraProxy.

Described new logging formats.

Updated description of Key management approach we encourage you to use.

Described Docker components and ready-to-use Docker Compose configurations based on the Docker Readme.

Updated Getting started with Docker guide.

Distributed the information about master key across the docs.

Many small improvements.

Source code(tar.gz)
Source code(zip)
0.76(Mar 9, 2018)
Core:

SSL / TLS support

Now you can use PostgeSQL with SSL/TLS settings enabled. Acra supports two modes of connection between AcraServer and the database: using SSL/TLS or using Secure Session (#113, #119).

Unix sockets

Acra now supports usage of both TCP and Unix Sockets as a connection layer between AcraWriter <-> AcraProxy <-> AcraServer.

Tests

Updated integration test suit to support multiple connection modes between the Acra components and the database (#115, #117, #118, #120).

Added Docker image to make testing easier (#104).

Infrastructure:

Added support of Go 1.10, removed support of older Go versions (<1.6).

Added support of Ubuntu 17.10, Ubuntu 16.04, Ubuntu 14.04, Debian Stretch.

Updated dependency libraries (libthemis and libcrypto) to use the latest ones.

Documentation:

Updated the documentation and tutorials to reflect the latest changes.

Source code(tar.gz)
Source code(zip)
0.75(Mar 7, 2017)
This is the initial public release of Acra, a database protection suite.

This version of Acra:

works on Ubuntu, CentOS, Debian linuxes

supports PostgreSQL 9.4+

has AcraWriter packages for Python, PHP, Go and NodeJS

Source code(tar.gz)
Source code(zip)
acra-0.75-1.i686.rpm(13.35 MB)
acra-0.75-1.x86_64.rpm(14.18 MB)
acra-0.75-amd64.deb(6.61 MB)
acra-0.75-i386.deb(5.92 MB)