This is part twenty-five in a series on the language spec for the Merg-E Domain Specific Language for the InnuenDo Web 3.0 stack. This post refers to both version v0.3 and version v0.4 of the language spec. I'll add more parts to the below list as the spec progresses:
- part 1 : coding style, files, merging, scoping, name resolution and synchronisation
- part 2 : reverse markdown for documentation
- part 3 : Actors and pools.
- part 4 : Semantic locks, blockers, continuation points and hazardous blockers
- part 5 : Semantic lexing, DAGs, prune / ent and alias.
- part 6 : DAGs and DataFrames as only data structures, and inline lambdas for pure compute.
- part 7 : Freezing
- part 8 : Attenuation, decomposition, and membranes
- part 9 : Sensitive data in immutables and future vault support.
- part 10 : Scalars and High Fidelity JSON
- part 11 : Operators, expressions and precedence.
- part 12 : Robust integers and integer bitwidth generic programming
- part 13 : The Merg-E ownership model, capture rules, and the --trustmebro compiler flag.
- part 14 : Actorcitos and structural iterators
- part 15 : Explicit actorcitos, non-inline structural iterators, runtimes, and abstract scheduler pipeline.
- part 16 : async functions and resources and the full use of InnuenDo VaultFS
- part 17: RAM-Points, RAM-points normalization bag, and the quota-membrane.
- part 18: Literal operators & Rational and Complex numbers.
- part 19: Interaction between operators, integer bitwidth generics, and the full numeric type-system.
- part 20 (v0.4): Compile-time dimensional analysis, SI/Planck units and the scaling literal operator.
- part 21 (v0.4): Tensors and tensor literals.
- part 22 (v0.4): Deprecating float/complex for rquantity/cquantity for full dimensional type-safety.
- part 23 : Flow control and exceptions.
- part24: The merge parts of Merg-E
- part25: The $threshold keyword and logging.
In this post we do something that I don't take lightly, and that I hope won't be needed again. We extend the Merg-E language with a third keyword. So far we managed to define the entire language with just two keywords, $scope and $resolve_order, and today we add the new keyword $threshold to that still pretty short list. And why do we need to add this new keyword? The answer to that question is for zero overhead logging. It is likely the keyword may come in handy for other purposes too, but the reason I took the big step of adding a third keyword to the language is logging.
Logging and log levels
If you have ever used log APIs in other la/hive-139531/@pibara/version-03-of-the-merg-e-language-specification--the-merge-parts-of-merg-enguages, chances are you are familiar with log levels.
Usually logging is not an intrinsic part of the language, and neither are log levels like DEBUG, NOTICE, ERROR and ALERT. It is up to the implementation of the logging library to make DEBUG logging be as efficient at doing nothing then not running in or compiled for debug mode. Languages like C++ have some nifty tricks up their sleeves to make most logging code at such levels evaporate, but Merg-E as for now does not.
Separating levels and thresholds from logging.
So how can we solve this problem for Merg-E ? We do want the ability to have verbose debug logs when debugging, but we don't want to pay the runtime CPU-usage price paid by all ambient resource use in Merg-E. We choose to do this by separating the log levels from the ambient log resource, and to do that we need the new keyword.
While logging is the justifying reason for the existence of the keyword, we choose to implement it in an extensible way for future language updates.
Because almost everything in Merg-E is a tree-like graph, and we already do the same for the $scope keyword, we treat $threshold like a tree-like path anchor as well.
Basically any $threshold expression means:
"erase the following Merg-E expression unless the compiletime set threshold reaches this high"
$threshold.log.debug log "starting merengue service" endl;
As we discussed in other posts a few times already, the Merg-E compilation pipeline consists of multiple phases, starting with bones, that only creates a first stage parse tree. The $threshold keyword only exists in the bones phase. After the bones phase, no later compiler stage ever sees a $threshold expression. It has either been erased completely or rewritten into the underlying expression.
We define a new command line flag --thresholds that defines how the above $threshold expression is processed. The processing is extremely simple.
merg-e-bones --thresholds log.warning myapp/myapp.mrg build/myapp-bones.json
If we call bones like this, the whole entire expression simply disappears from the bones parse-tree output. It would be as if the line wasn't in the code.
But when we change it to :
merg-e-bones --thresholds log.debug myapp/myapp.mrg build/myapp-bones.json
The line gets processed as if $threshold.log.debug wasn't there and the code looked like this:
log "starting merengue service" endl;
Comma vs semicolon
As discussed before, inside of a capture spec and use expressions, commas and semicolons are interchangeable as separator between captures. But this equivalence changes with the presence of $threshold. We define that a semicolon ends an expression for $threshold while a comma does not.
mutable main ()::{
$threshold.log.warning log;
$threshold.log.warning log_endl}{
...
In this example main captures both log and log_endl from the application source scope, but it does so using $threshold twice. Alternatively we can use the idiomatic comma form and not duplicate the threshold:
mutable main ()::{
$threshold.log.warning log,
log_endl}{
...
thresholds and least authority
As hinted at in the previous section, threshold usage weaves in very much into the least authority fabric of the Merg-E language. Let us use a slightly updated version of some sample code we used before. Take note at all five places where we use $reshold.
app myApp lang ambient {
use ambient {
$threshold.log.error logging.default.log as log,
logging.endl as log_endl
};
mutable main ()::{
$threshold.log.error log,
log_endl}{
inert whole16 max_prime = 1000;
shared mutable whole16 ok_count = 0;
mutable blocker all_primes;
reentrant mutable merge utils.is_prime as is_prime(x whole16)::{
ok_count;
$threshold.log.error log,
log_endl
}{
max_prime;
}@[range_error];
take 100 {
all_primes += is_prime($scope.current + 1);
}
await_all all_primes;
$threshold.log.info log "counted " ok_count " prime numbers" log_endl;
}!!{
range_error : {
$threshold.log.error log "Something went wrong" log_endl;
}};
}
The last two uses are obvious. If the threshold is info or higher we log the number of prime numbers found. If the threshold is error or higher we log range error exceptions.
The $threshold within the merge expression changes the least authority contract between main and the is_prime callable from the utils module depending on the threshold. If the threshold is error or higher, log and log_end are made part of the capture part of the merg contract, if it isn't, then is_prime won't need access to log or log_endl.
Because the merge expression and the error body needs error or higher and the prime count logging needs info or higher, given that info is higher than error, main needs error, and we can thus use another threshold in the capture spec for main using $threshold.log.error. And if we don't need log and log_endl, we don't need the use expression in the application body to include them unless the threshold is error or higher either, so we put an other $threshold prefix in the use clause.
We see how the threshold influences the least authority propagation of logging or other ambient resources though the program and our modules,
Because authority propagation is derived from the parse tree after threshold elimination, builds compiled with a lower threshold automatically possess less ambient authority.
future namespaces
For now $threshold.log is the only namespace we define in our bones implementation, but future versions of the language are very likely to have other namespaces available too. Think for example of things like profiling levels or the absence of presence of runtime asserts. Note that these are just two examples of what might be, no concrete plans for other namespaces currently exist.
conclusion
In this post we discussed the new third keyword in the Merg-E language threshold. It was a hard decision to add a third keyword to a language that prides itself on its minimum amount of keywords, but with a lack of fancy tricks that could have made debug logging code close to evaporate at compile time for non-debug builds, something intrusive was called for. By choosing to separate the threshold functionality from logging, threshold conditional expressions are not just a logging tool, they are a tool that improves Merg-E as a least authority language. Now logging is just one optional ambient resource that we may or may not need depending on threshold values. For now we only facilitate logging with it, but in the future we expect other language features and ambient resources to get their own namespace for threshold usage.
