Oak is an expressive, dynamically typed programming language

Overview

Oak 🌳

Build Status

Oak is an expressive, dynamically typed programming language. It takes the best parts of my experience with Ink, and adds what I missed and removes what didn't work to get a language that feels just as small and simple, but much more ergonomic and capable.

Here's an example Oak program.

std := import('std')

fn fizzbuzz(n) if [n % 3, n % 5] {
    [0, 0] -> 'FizzBuzz'
    [0, _] -> 'Fizz'
    [_, 0] -> 'Buzz'
    _ -> string(n)
}

std.range(1, 101) |> std.each(fn(n) {
    std.println(fizzbuzz(n))
})

Oak has good support for asynchronous I/O. Here's how you read a file and print it.

std := import('std')
fs := import('fs')

with fs.readFile('./file.txt') fn(file) if file {
    ? -> std.println('Could not read file!')
    _ -> print(file)
}

Oak also has a pragmatic standard library that comes built into the oak executable. For example, there's a built-in HTTP server and router in the http library.

std := import('std')
fmt := import('fmt')
http := import('http')

server := http.Server()
with server.route('/hello/:name') fn(params) {
    fn(req, end) if req.method {
        'GET' -> end({
            status: 200
            body: fmt.format('Hello, {{ 0 }}!'
                std.default(params.name, 'World'))
        })
        _ -> end(http.MethodNotAllowed)
    }
}
server.start(9999)

Overview

Oak has 7 primitive and 3 complex types.

?        // null, also "()"
_        // "empty" value, equal to anything
1, 2, 3  // integers
3.14     // floats
true     // booleans
'hello'  // strings
:error   // atoms

[1, :number]    // list
{ a: 'hello' }  // objects
fn(a, b) a + b  // functions

These types mostly behave as you'd expect. Some notable details:

  • There is no implicit type casting between any types, except during arithmetic operations when ints may be cast up to floats.
  • Both ints and floats are full 64-bit.
  • Strings are mutable byte arrays, also used for arbitrary data storage in memory, like in Lua. For immutable strings, use atoms.
  • Lists are backed by a vector data structure -- appending and indexing is cheap, but cloning is not
  • For lists and objects, equality is defined as deep equality. There is no identity equality in Oak.

We define a function in Oak with the fn keyword. A name is optional, and if given, will define that function in that scope. If there are no arguments, the () may be omitted.

fn double(n) 2 * n
fn speak {
    println('Hello!')
}

Besides the normal set of arithmetic operators, Oak has a few strange operators.

  • The assignment operator := binds values on the right side to names on the left, potentially by destructuring an object or list. For example:

    a := 1              // a is 1
    [b, c] := [2, 3]    // b is 2, c is 3
    d := double(a)      // d is 2
  • The nonlocal assignment operator <- binds values on the right side to names on the left, but only when those variables already exist. If the variable doesn't exist in the current scope, the operator ascends up parent scopes until it reaches the global scope to find the last scope where that name was bound.

    n := 10
    m := 20
    {
        n <- 30
        m := 40
    }
    n // 30
    m // 20
  • The push operator << pushes values onto the end of a string or a list, mutating it, and returns the changed string or list.

    str := 'Hello '
    str << 'World!' // 'Hello World!'
    
    list := [1, 2, 3]
    list << 4
    list << 5 << 6 // [1, 2, 3, 4, 5, 6]
  • The pipe operator |>, which takes a value on the left and makes it the first argument to a function call on the right.

    // print 2n for every prime n in range [0, 10)
    range(10) |> filter(prime?) |>
        each(double) |> each(println)
    
    // adding numbers
    fn add(a, b) a + b
    10 |> add(20) |> add(3) // 33

Oak uses one main construct for control flow -- the if match expression. Unlike a traditional if expression, which can only test for truthy and falsy values, Oak's if acts like a sophisticated switch-case, comparing values until the right match is reached.

fn pluralize(word, count) if count {
    1 -> word
    2 -> 'a pair of ' + word
    _ -> word + 's'
}

This match expression, combined with safe tail recursion, makes Oak Turing-complete.

Lastly, because callback-based asynchronous concurrency is common in Oak, there's special syntax sugar, the with expression, to help. The with syntax sugar de-sugars like this.

with readFile('./path') fn(file) {
    println(file)
}

// desugars to
readFile('./path', fn(file) {
    println(file)
})

For a more detailed description of the language, see the work-in-progress language spec.

Builds and deployment

While the Oak interpreter can run programs and modules directly from source code on the file system, Oak also offers a build tool, oak build, which can bundle an Oak program distributed across many files into a single "bundle" source file. oak build can also cross-compile Oak bundles into JavaScript bundles, to run in the browser or in JavaScript environments like Node.js and Deno. This allows Oak programs to be deployed and distributed as single-file programs, both on the server and in the browser.

To build a new bundle, we can simply pass an "entrypoint" to the program.

oak build --entry src/main.oak --output dist/bundle.oak

Compiling to JavaScript works similarly, but with the --web flag, which turns on JavaScript cross-compilation.

oak build --entry src/app.js.oak --output dist/bundle.js --web

The bundler and compiler are built on top of my past work with the September toolchain for Ink, but slightly re-architected to support bundling and multiple compilation targets. In the future, the goal of oak build is to become a lightly optimizing compiler and potentially help yield an oak compile command that could package the interpreter and an Oak bundle into a single executable binary. For more information on oak build, see oak help build.

Performance

As of September 2021, Oak is about 5-6x slower than Python 3.9 on pure function call and number-crunching overhead (assessed by a basic fib(30) benchmark). These figures are worst-case estimates -- because Oak's data structures are far simpler than Python's, the ratios start to go down on more realistic complex programs. But nonetheless, this gives a good estimate of the kind of performance (or, currently, the lack thereof) you can expect from Oak programs. It's not fast, though anecdotally it's fast enough for me to have few complaints for most of my use cases.

Runtime performance is not currently my primary concern; my primary concern is implementing a correct and pleasant interpreter that's fast enough for me to write real apps with. Only when speed becomes a problem for software I built with Oak will I really invest much more in speed. I think being as fast as Python and Ruby is a good goal, long-term. Those languages run in production and receive continuous investments into performance tuning, but are far more complex. Oak is much simpler, but it's also just me. I think it evens out the difference.

There are several immediately actionable things we can do to speed up Oak programs' runtime performance, though none are under works today. In order of increasing implementation complexity:

  1. Basic compiler optimization techniques applied to the abstract syntax tree, like constant folding and propagation.
  2. Variable name mangling and caching, so string comparisons become u64 comparisons between machine words.
  3. A thorough audit of the interpreter's memory allocation profile and a memory optimization pass (and the same for L1/L2 cache misses).
  4. A bytecode VM that executes Oak compiled down to more compact and efficient bytecode rather than a syntax tree-walking interpreter.

Development

Oak (ab)uses GNU Make to run development workflows and tasks.

  • make run compiles and runs the Oak binary, which opens an interactive REPL
  • make fmt or make f runs the oak fmt code formatter over any files with unstaged changes in the git repository. This is equivalent to running oak fmt --changes --fix.
  • make tests or make t runs the Go tes suite for the Oak language and interpreter
  • make test-oak or make tk runs the Oak test suite, which tests the standard libraries
  • make test-bundle runs the Oak test suite, bundled using oak build
  • make test-js runs the Oak test suite on the system's Node.js, compiled using oak build --web
  • make install installs the Oak interpreter on your $GOPATH as oak, and re-installs Oak's vim syntax file

To try Oak by building from source, clone the repository and run make install (or simply go build).

Unit and generative tests

The Oak repository so far as two kinds of tests: unit tests and generative/fuzz tests. Unit tests are just what they sound like -- tests validated with assertions -- and are built on the libtest Oak library with the exception of Go tests in eval_test.go. Generative tests include fuzz tests, and are tests that run some pre-defined behavior of functions through a much larger body of procedurally generated set of inputs, for validating behavior that's difficult to validate manually like correctness of parsers and libdatetime's date/time conversion algorithms.

Both sets of tests are written and run entirely in the "userland" of Oak, without invoking the interpreter separately. Unit tests live in ./test and are run with ./test/main.oak; generative tests are in test/generative, and can be run manually.

Issues
  • The name

    The name "oak" may conflict with another established language

    oak is another programming language named oak and from what I can tell is a little more established.

    Is there a specific way to you'd want this language referred to in areas where a conflict may arise? ( package managers, projects similar to rosettacode, etc.. )

    opened by brecert 0
  • JS Interop?

    JS Interop?

    Given that oak can be transpiled to js, is it:

    1. Possible,
    2. Desirable given the goals of the project

    to allow interop with js libraries?

    (Sorry if this is the wrong place to put this - if this is better asked in another place or a discussion thread, please let me know 🙇‍♂️)

    opened by shrik450 0
  • Linus's myriad bug list

    Linus's myriad bug list

    Language & CLI

    • [x] std.{is, constantly} for more fluent code, as in Klisp
    • [x] std.{exclude, separate} to complement std.filter and enable more fluent code, as in Klisp
    • [x] path.resolve does not fully resolve /./ and /../ patterns (collapse those paths down) if the input path is absolute.
    • [x] str.{rindex, rfind} for searching a string from the right, similar to Python's rfind and friends.
    • [x] A function in the http library to parse, construct, and manipulate query parameter strings, e.g. { a: true, b: 23 } <=> a=true&b=23. Note this should take into account http.percentEncode. It should omit null values and JSON-serialize any composite values.
    • [x] str.space? (and therefore 1-adic forms of str.trim and friends) are broken on --web for the \r character.
    • [x] Variants of std.loop that I can use with async loop bodies for an async loop (e.g. reading a file into a fixed-size buffer, concurrent jobs working from a list of file paths). This will come in three versions, an aloop for unbounded async looping as well as a serial and a parallel variant which facilitate serial and parallel processing respectively.
    • [ ] str.rindexOf to match std.rindexOf
    • [ ] TypeScript-style static typing with type inference (https://github.com/Ahnfelt/type-inference-by-example), perhaps in a built-in CLI tool like oak check. Teal (https://github.com/teal-language/tl) is another effective inspiration.

    Website

    • [x] Download link on the website uses a relative url #start instead of an absolute /#start, so is broken on any page that isn't the main page.
    • [x] Highlight proxy is broken for URLs with query parameters, like links to private GitHub repo source files, because the query parameters aren't encoded properly in the proxy <form> and therefore fed to the proxy as query params to the proxy, not the original proxied URL.
    • [x] Write some quick blog about the codecols utility
    bug 
    opened by thesephist 0
Releases(v0.2)
  • v0.2(Mar 15, 2022)

    There's been a huge wave of improvements, fixes, and additions since v0.1 less than two months ago. The highlights include a static site generation system for the Oak website, expanded standard library modules, a much-improved and optimized Oak-to-JavaScript compiler, a built-in syntax highlighter, and oak eval / oak pipe commands.

    std := import('std')
    Oak := { version: '0.1' }
    Oak |> std.merge({ version: '0.2' })
    

    Standard library additions

    • std.(take, takeLast) in the standard 'std' module
    • std.stdin to read all of standard input in the 'std' module
    • std.loop can now return meaningful values from within the loop
    • str.trimStart and str.trimEnd now have forms with optional second argument, which will trim all whitespace to match the behavior of str.trim
    • sqrt, hypot, scale, bearing, orient, prod, clamp in the standard 'math' module
    • Statistics functions mean, median, stddev in the standard 'math' module
    • t.approx for comparing floating-point values in the 'test' module
    • fmt.format now also accepts format string variables that key into objects, rather than integer-index into the argument list. This means expressions like fmt.format('{{ name }}: {{ age }}, userProfile) work.

    Improvements

    • Boolean combinators & and | now short-circuit in Oak
    • oak cat for syntax highlighting Oak files in the command line
    • oak eval and oak pipe for evaluating Oak code from the UNIX command line
    • The try() built-in function for interoperability with JavaScript's checked exceptions
    • Many rounds of optimizations went into the oak build --web compiler, including a few passes focused on bundle size.
    • The oak binary can now evaluate Oak code from standard input.
    • The router in the 'http' module now accepts any HTTP header, including OPTIONS, HEAD, and others.
    • The standard testing module 'test' how uses debug.inspect to pretty-print outputs of failing tests for more readable test results.
    • Tokens from syntax.Tokenizer now contain their byte offset position in the source file, in addition to line and column numbers.
    • The input() built-in function now reads from a single global buffer, and is capable of reading large files from standard input.

    Fixes

    • Fixed infinite recursion from circular imports in bundles generated by oak build.
    • Deleting keys with o.key := _ now always works in JavaScript contexts.
    • str.(lower, upper) are more idiomatic and faster.
    • str.(upper?, lower?, digit?, space?, startsWith?, endsWith?) are faster.
    • Fix malformed format strings crashing fmt.format.
    • Guard against XSS vulnerabilities in the 'md' standard library module.
    • oak build --web now uses the JavaScript null coalescing operator ?? for safer and smaller bundles.
    • Fix some parser crashes caused by malformed Oak syntax input.
    • Improve error messages from syntax.Parser to always include the syntax error position.
    • The req() built-in function now correctly returns an error event instead of crashing if the HTTP connection fails to establish.

    oaklang.org is live!

    The Oak website has gotten to a point where I can officially say it's not "incomplete". There's still not much documentation on the site about the language or the standard libraries, but there is:

    Source code(tar.gz)
    Source code(zip)
    oak-darwin(5.60 MB)
    oak-linux(5.27 MB)
  • v0.1(Jan 23, 2022)

    This is the first release of the Oak language and Oak CLI, which includes not only the interpreter, but also a code formatter, a bundler, and an Oak-to-JavaScript compiler. 🌳

    std := import('std')
    std.println('Hello, World!')
    

    Although the full language and standard library are implemented and tested in both the interpreter and the compiler, the language and the standard library may yet change in unexpected ways as I further exercise them in my day to day work. There may also be un-tested edge cases in both that merit further changes in semantics for those edge cases. Note, also, that there's... basically no documentation yet on the language or the stdlib, so if you want to venture forth and try Oak, you'll have to read the standard library to see both how to use Oak, and how the standard library works.

    Nonetheless, Oak from this release can run and compile any of the samples in this repository or Oak codebases elsewhere on GitHub as of today!

    Source code(tar.gz)
    Source code(zip)
    oak-darwin(5.58 MB)
    oak-linux(5.26 MB)
Owner
Linus Lee
Languages, interpreters, compilers. Building better tools and workflows, usually on the Web, sometimes for the Web.
Linus Lee
Advent of Code is an Advent calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved in any programming language you like.

Advent of Code 2021 Advent of Code is an Advent calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved

Kemal Ogun Isik 0 Dec 2, 2021
Zach Howell 0 Jan 4, 2022
Unit tests generator for Go programming language

GoUnit GoUnit is a commandline tool that generates tests stubs based on source function or method signature. There are plugins for Vim Emacs Atom Subl

Max Chechel 61 Jun 21, 2022
FreeSWITCH Event Socket library for the Go programming language.

eventsocket FreeSWITCH Event Socket library for the Go programming language. It supports both inbound and outbound event socket connections, acting ei

Alexandre Fiori 106 Jun 24, 2022
Simple interface to libmagic for Go Programming Language

File Magic in Go Introduction Provides simple interface to libmagic for Go Programming Language. Table of Contents Contributing Versioning Author Copy

Krzysztof Wilczyński 12 Dec 22, 2021
The Gorilla Programming Language

Gorilla Programming Language Gorilla is a tiny, dynamically typed, multi-engine programming language It has flexible syntax, a compiler, as well as an

null 29 Apr 16, 2022
Elastic is an Elasticsearch client for the Go programming language.

Elastic is an Elasticsearch client for the Go programming language.

Oliver Eilhard 6.9k Jun 26, 2022
👩🏼‍💻A simple compiled programming language

The language is written in Go and the target language is C. The built-in library is written in C too

paco 26 May 26, 2022
Lithia is an experimental functional programming language with an implicit but strong and dynamic type system.

Lithia is an experimental functional programming language with an implicit but strong and dynamic type system. Lithia is designed around a few core concepts in mind all language features contribute to.

Valentin Knabel 6 Jun 14, 2022
accessor methods generator for Go programming language

accessory accessory is an accessor generator for Go programming language. What is accessory? Accessory is a tool that generates accessor methods from

masaushi 4 Apr 30, 2022
Http web frame with Go Programming Language

Http web frame with Go Programming Language

jk 0 Oct 17, 2021
A modern programming language written in Golang.

MangoScript A modern programming language written in Golang. Here is what I want MangoScript to look like: struct Rectangle { width: number he

PlebusSupremus1234 3 Nov 12, 2021
A stack oriented esoteric programming language inspired by poetry and forth

paperStack A stack oriented esoteric programming language inspired by poetry and forth What is paperStack A stack oriented language An esoteric progra

null 0 Nov 14, 2021
Besten programming language

Besten What holds this repository? Besten Lexer Besten Parser Besten Interpreter Besten Module Loader Besten Lexer Located in ./internal/lexer A set o

Alberto Elorza 2 Jun 13, 2022
🎅 A programming language for Advent of Code.

?? Adventlang My blog post: Designing a Programming Language for Advent of Code A strongly typed but highly dynamic programming language interpreter w

Andrew Healey 27 Mar 2, 2022
An experimental programming language.

crank-lang An experimental & interpreted programming language written in Go. Features C like syntax Written in Golang Interpreted Statically Typed Dis

inmicro 2 Dec 6, 2021
Gec is a minimal stack-based programming language

Gec is a minimal stack-based programming language

aiocat 1 May 22, 2022
Functional programming library for Go including a lazy list implementation and some of the most usual functions.

functional A functional programming library including a lazy list implementation and some of the most usual functions. import FP "github.com/tcard/fun

Toni Cárdenas 31 May 21, 2022
Flow-based and dataflow programming library for Go (golang)

GoFlow - Dataflow and Flow-based programming library for Go (golang) Status of this branch (WIP) Warning: you are currently on v1 branch of GoFlow. v1

Vladimir Sibirov 1.4k Jun 24, 2022