Oren Eini

CEO of RavenDB

a NoSQL Open Source Document Database

Get in touch with me:

oren@ravendb.net +972 52-548-6969

Posts: 7,583
|
Comments: 51,213
Privacy Policy · Terms
filter by tags archive
time to read 1 min | 180 words

You can register here for a two days course in building DSL with Boo.

It is going to take place two weeks from today, in Austin. (19 - 20 May)

I know that this is short notice, but it wasn't something that was planned well in advance. It came out of the ALT.Net conference.

Topics:

  • Creating Domain Specific Languages
  • The Boo Language
  • Flexible compiler and malleable language
  • Creating applications with embedded DSL
  • Management, tracing and debugging
  • Tooling support
  • Testing and maintainability concerns

There are ten seats open for that.

I hope we would have fun.

I would also like to thank Jeffrey Palermo and Headspring for hosting the course.

time to read 1 min | 180 words

// We create a class for the macro, the class name is
// meaningful, [macro name]Macro allows us to later refer
// to the macro using just [macro name].
// Note that we inherit from AbstractAstMacro
class UnrollMacro(AbstractAstMacro):

	// Here we perform the actual compiler manipulation
	// the compiler hands us a macro statement, and we have
	// to return a statement back, which will replace it.
	override def Expand(macro as MacroStatement) as Statement:
	
		// define a block of code
		block = Block()
		
		// extract the second parameter value
		end = cast(IntegerLiteralExpression, macro.Arguments[1]).Value
		
		for i in range(end):
			// create assignment statement, using the block: trick 
			// and add it to the output
			statements = [|
				block:
					$(macro.Arguments[0]) = $i
			|].Block
			block.Add(statements)
			
			// add the original contents of the macro
			// to the output
			block.Add(macro.Block)
			
		return block
And usage:
unroll i, 5:
	print i

Which will produce:

i = 0
print i
i = 1
print i
i = 2
print i
i = 3
print i
i = 4
print i
time to read 5 min | 813 words

I am doing some work on the DSL book right now, and I run into this example, which is simple too delicious not to post about.

Assume that you have the following UI, which you use to let a salesperson generate a quote for your system.

image

This is much more than just a UI issue, to be clear. You have fully fledged logic system here. Calculating the total cost is the easy part, first you have to understand what you need.

Let us define a set of rules for the application, is will be clearer when we have the list in front of us:

  • The Salary module requires a machine per every 150 users.
  • The Taxes module requires a machine per 50 users.
  • The Vacations module requires the Scheduling Work module.
  • The Vacations module requires the External Connections module.
  • The Pension Plans module requires the External Connections module.
  • The Pension Plans module must be on the same machine as the Health Insurance module.
  • The Health Insurance module requires the External Connections module.
  • The Recruiting module requires a connection to the internet, and therefore requires a fire wall of the recommended list.
  • The Employee Monitoring module requires the CompMonitor component

Of course, this fictitious sample is still too simple, we can probably sit down and come up with fifty or so more rules that we need to handle. Just handling the second level dependencies (External Connections, CompMonitor, etc) would be a big task, for example.

Assume that you have not a single such system, but 50 of them. I know of a company that spent 10 years and has 100,000 lines of C++ code (that implements a poorly performing Lisp machine, of course) to solve this issue.

My solution?

specification @vacations:
	requires @scheduling_work
	requires @external_connections
	
specification @salary:
	users_per_machine 150
	
specification @taxes:
	users_per_machine 50

specification @pension:
	same_machine_as @health_insurance

Why do we need a DSL for this? Isn’t this a good candidate for data storage system? It seems to me that we could have expressed the same ideas with XML (or a database, etc) just as easily. Here is the same concept, now express in XML.

<specification name="vacation">
	<requires name="scheduling_work"/>
	<requires name="external_connections"/>
</specification>

<specification name="salary">
	<users_per_machine value="150"/>
</specification>

<specification name="taxes">
	<users_per_machine value="50"/>
</specification>

<specification name="pension">
	<same_machine_as name="health_insurance"/>
</specification>

That is a one to one translation of the two, why do I need a DSL here?

Personally, I think that the DSL syntax is nicer, and the amount of work to get from a DSL to the object model is very small compared to the work required to translate to the same object model from XML.

That is mostly a personal opinion, however. For pure declarative DSL, we are comparable with XML in almost all things. It gets interesting when we decide that we don’t want this purity. Let us add a new rule to the mix, shall we?

  • The Pension Plans module must be on the same machine as the Health Insurance module, if the user count is less than 500.
  • The Pension Plans module requires distributed messaging backend, if the user count is great than 500.

Trying to express that in XML can be a real pain. In fact, it means that we are trying to shove programming concepts into the XML, which is always a bad idea. We could try to put this logic in the quote generation engine, but that is complicating it with no good reason, tying it to the specific application that we are using, and in general making a mess.

Using our DSL (with no modification needed), we can write it:

specification @pension: 
	if information.UserCount < 500: 
		same_machine_as @health_insurance 
	else: 	
		requires @distributed_messaging_backend

As you can imagine, once you have run all the rules in the DSL, you are left with a very simple problem to solve, with all the parameters well known.

In fact, throughout the process, there isn't a single place of overwhelming complexity.

I like that.

time to read 1 min | 92 words

Just found myself writing that, and it was amusing.

import System.Net
import System.IO

if argv.Length != 2:
	print "You must pass [prefix] [path] as parameters"
	return

prefix = argv[0]
path = argv[1]

if not Directory.Exists(path):
	print "Could not find ${path}"
	return

listener = HttpListener()
listener.Prefixes.Add(prefix)
listener.Start()

while true:
	context = listener.GetContext()
	file = Path.GetFileName(context.Request.RawUrl)
	fullPath = Path.Combine(path, file)
	if File.Exists(fullPath):
		context.Response.AddHeader("Content-Disposition","attachment; filename=${file}")
		bytes = File.ReadAllBytes(fullPath)
		context.Response.OutputStream.Write(bytes, 0, bytes.Length)
		context.Response.OutputStream.Flush()
		context.Response.Close()
	else:
		context.Response.StatusCode = 404
		context.Response.Close()
time to read 1 min | 81 words

One of the feedback points that I got about the book was that there need to be a fully working Boo program, before we start to introduce the DSL ones. The problem? I can't think of anything that will be both interesting and short enough to put into a book.

My limit is at about two pages, and I don't want a tedious example. Any suggestions?

One major point, it should have as little background assumptions as possible.

time to read 2 min | 225 words

Now, before Greg hurls a modopt on me, I want to be clear that this isn't the same thing that Spec# is doing. But it is a very cool way to specify constraints that must always be valid when a method exists.

Here is the code:

[AttributeUsage(AttributeTargets.Class)]
class EnsureAttribute(AbstractAstAttribute):
	
	expr as Expression
	
	def constructor(expr as Expression):
		self.expr = expr
		
	def Apply(target as Node):
		type as ClassDefinition = target
		for member in type.Members:
			method = member as Method
			continue if method is null
			block = method.Body
			method.Body = [|
				block:
					try:
						$block
					ensure:
						assert $expr
			|].Block

And the usage:

[ensure(name is not null)]
class Customer:
	name as string
		
	def constructor(name as string):
		self.name = name
	
	def SetName(newName as string):
		name = newName

Now, any attempt to set the name to null will cause an assertion exception. This technique is quite powerful, and very easy to use. A few years ago I wrote a design by contract implementation for boo that was far more ambitious (handling inheritance, etc). I remember it being much more complicated, and while things like quasi quotation do make it easier, it is not that big a change.

I think that mostly it is the way I write code now, striving to simplicity is something that I am trying to apply recently, and I think it works.

Meta Methods

time to read 3 min | 462 words

A meta-method is a shortcut into the compiler; it is a method that accepts AST nodes[1] and returns an AST node.

Let us implement this very simple scenario, the assert statement. Now, because Boo already has that, we will use “verify” as the method name. Here is the full method implementation:

[Meta]
static def verify(expr as Expression):
	return [|
		unless $expr:
			raise $(expr.ToCodeString())
	|]

We are using quasi quotation to save us typing. This is a static method decorated with the [Meta] attribute, and accepting an AST expression. This is all you need in order to create a meta-method. When you have a meta-method, you can call it, like this:

verify 1 == 2

Now the interesting tidbit happens. When the compiler sees a call to a meta-method, it doesn’t emit the code to call this method at runtime. Instead, during compilation, the meta-method is executed. We pass it the AST of the arguments of the method code (including anonymous blocks), and then we replace this method call with the result of calling the meta-method.

It is important that you’ll understand that after compilation, where in the code we had this:

verify 1 == 2

The actual compiled bits will have this:

unless 1 == 2:
	raise “1 == 2”

Please go over it again, to make sure that you understand how it works. It is similar to text substitution macros in C and C++, but this is actual code that is running during compilation that gets to output any code that it wants back into the compilation process, not mere text preprocessing. In addition to that, we are dealing directly with the compiler’s AST, not just copying lines of text.

This seems to be something that a lot of people have a hard time grasping. The compiler will ask you, at compilation time, what kind of transformation you want to do on the code. It will then take the result of the transformation (the method return value) and put it where the method call used to be.

The Boo code above can also be translated to the following C#, which is a bit more explicit about what is going on:

[Meta]
public static UnlessStatement verify(Expression expr)
{
	UnlessStatement unlessS = new UnlessStatement();
	unless.Condition = Expression.Lift(expr);
	RaiseStatement raise = new RaiseStatement();
	raise.Exception = Expression.Lift(expr.ToCodeString());
	unless.Statements.Add(raise);
	return unless;
}

Both have the same exact semantics.

We have actually used meta-methods before, when we implemented the “when” keyword for the scheduling DSL. Meta-methods are used in DSL quite often. They are usually the first step that we need to take into the compiler when we run into the limits of what the compiler gives us out of the box.


[1] An ast node is generic term to all the types that compose the abstract syntax tree of the language.

FUTURE POSTS

No future posts left, oh my!

RECENT SERIES

  1. Production postmorterm (2):
    11 Jun 2025 - The rookie server's untimely promotion
  2. Webinar (7):
    05 Jun 2025 - Think inside the database
  3. Recording (16):
    29 May 2025 - RavenDB's Upcoming Optimizations Deep Dive
  4. RavenDB News (2):
    02 May 2025 - May 2025
  5. Production Postmortem (52):
    07 Apr 2025 - The race condition in the interlock
View all series

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats
}