Oren Eini

CEO of RavenDB

a NoSQL Open Source Document Database

Get in touch with me:

oren@ravendb.net +972 52-548-6969

Posts: 7,583
|
Comments: 51,213
Privacy Policy · Terms
filter by tags archive
time to read 5 min | 910 words

Then it must be an IQuackFu.

IQuackFu is Boo’s answer to the Method Missing / Message Not Understood from dynamic languages. Since Boo is a statically typed language[1], and since method missing is such a nice concept to have, we use this special interface to introduce this capability.

You are probably confused, because I didn’t even explain what method missing is. Let us go back and look at an example, shall we? We want to look at the following xml:

<People> 	<Person> 		<FirstName>John</FirstName> 	</Person> 	<Person> 		<FirstName>Jane</FirstName> 	</Person> </People> 

Now we want to display the first names in the xml. We can do it using XPath, but the amount of code required makes this awkward. We can also generate some sort of strongly typed wrapper around it, assuming that we have a schema for this, we can use a tool to generate the schema, if we don’t have it already…

Doesn’t it look like a lot of work? We can also do this:

doc = XmlObject(xmlDocument.DocumentElement)
for person as XmlObject in doc.Person:
print person.FirstName

But we are using a generic object here, how can this work? This works because we intercept the calls to the object and decide how to answer them at runtime. This is the meaning of the term “method missing”. We “catch” the method missing and decide to do something smart about it (like returning the data from the xml document).

At least, this is how it works in dynamic languages. For a statically typed language, the situation is a bit different; all method calls must be known at compile time. That is why Boo introduced the idea of IQuackFu. Let us check the implementation of XmlObject first, and then we will discuss how it works:

class XmlObject(IQuackFu):
_element as XmlElement

def constructor(element as XmlElement):
_element = element

def QuackInvoke(name as string, args as (object)) as object:
pass # ignored

def QuackSet(name as string, parameters as (object), value) as object:
pass # ignored

def QuackGet(name as string, parameters as (object)) as object:
elements = _element.SelectNodes(name)
if elements is not null:
return XmlObject(elements[0]) if elements.Count == 1
return XmlObject(e) for e as XmlElement in elements

override def ToString():
return _element.InnerText

We didn’t implement the QuackInvoke and QuackSet, because they are not relevant to the example at hand, I think that QuackGet will make the point. Now, just to complete the picture, we will write the first code sample, the use of XmlObject, as the compiler will output it.

doc = XmlObject(xmlDocument)
for person as XmlObject in doc.QuackGet(“Person”):
print person.QuackGet(“FirstName”)

The way it works, when the compiler finds that it can’t resolve a method (or a property) in the usual way, it then check if the type implements the IQuackFu interface. If it does implement IQuackFu, it translates the method call into the equivalent method call.

The example of the Xml Object is a really tiny one of the possibilities. Convention based methods are an interesting idea[2] that is widely used in Ruby. Here is an example that should be immediately familiar to anyone who dabbled in Rails’ ActiveRecord:

user as User = Users.FindByNameAndPassword(“foo”, “bar”)

Which will be translated by the compiler to:

user as User = Users.QuackInvoke(“FindByNameAndPassword”, “foo”, “bar”)

The Users’ QuackInvoke method will parse the “method name” and issue a query by name and password.

You can do some very interesting things with IQuackFu...


[1] Well, it is statically typed unless you explicitly tell the compiler that you want late bound semantics. Aside from working against IDispatch COM interfaces, I have rarely found that ability useful. One case I did find it useful, however, was when I wanted to introduce Context Parameters, which we will discuss in a few pages.

[2] For the adventurous sorts, you can also do something called Lazy Methods, in which you generate a method if and only if it is being called. This is an interesting exercise in extending the compiler, but for all intents and purposes, IQuackFu answers this need very well.

time to read 1 min | 75 words

When you are not sure how to do something in Boo, try doing it like you would with C# (with the obvious syntax changes), in most cases, it would work. It may not be the best way to do something, however.

Keep this a secret, I may get thrown out of the Boo Associated Hackers community if that would happen, and where I would be without my BAH! membership?

time to read 3 min | 445 words

I really like the CLR. It is a great platform, it has a rich set of libraries, it has a lot of power and flexibility, and it was designed with multi language support in mind, which means that what you can do on the CLR vs. what you can do in your language are two different things.

It is just too bad that the default CLR language is C#. Now, that probably raised a few brows, and definitely some hackles. C# is a great language, I can hear you saying. Well, yes, sort of. If you thinks that holding the compiler’s hands is a useful thing to do. Leaving aside the ability to extend the compiler, Boo has the following to offer us:

Syntactic sugar for common programming patterns - List, hash and array literals, object initialization, string formatting and regular expression matching are all first class concepts in Boo, with direct support for all of them in a natural manner.

Automatic Variable Declaration together with Automatic type inference – the compiler takes care of things for you, instead of having to type the same thing over and over and over again. Some would say that this is bad, for them I would reply that they should try it first. It works.
Take a look at this:

def random():
	return 4 # selected by dice roll, guaranteed to be random
val = random() 

Is there are reason that I would need to specify the type over and over again? The compiler can figure it out for itself and not bother you with it. If you want to get a compiler error, you can:

val as string = random() # will error about type mismatch 

Automatic type casting – don’t make me explicitly say it, figure it out for me. I’ll have unit tests to check to cover me.

Duck typing – Boo is a strongly typed language, but you can ask the compiler to relax those constraints at certain scenarios. It makes some things much more natural, especially since you have a way to get into this infrastructure and decide what to do at runtime.
If we will take a simple example, let us look at the XmlObject and what we can do with it:

person = XmlObject(xmlDocument)
print person.FirstName
print person.LastName 

What we just did was resolve, at runtime, that we were asked to get the value of a property called “FirstName”, and we did. This is a trivial piece of code to implement.

Just those benefits present a significant improvement in the language experience, and we haven’t even touched the extensible compiler yet.

The Boo Language

time to read 4 min | 729 words

So, what is this Boo language anyway? I am talking about it fairly often, but I never explained, and I am going to assume that people aren't following links :-)

image Boo is an object oriented, statically typed language for the common language runtime with a Python inspired syntax and focused on language and compiler extensibility.

Boo is an Open Source project, released under the BSD license. This means that you are free to take, modify and use the language and its products in any way you want, without any limitation[1]

. Rodrigo B. de Oliveira started the project, and since then grown in popularity. The project is active and continually evolving. At the time of the writing, the released version of Boo is 0.8.

However, Boo seems to be following the trend of many open source projects, in that that it is not really rushing toward 1.0 release. Boo has been around for a number of years, and I have been using it for production for the last two years.

Production, in this case, means systems that move money around with my neck on the line. I have a high degree of trust in it, obviously.

Beyond being a really malleable language, suitable for DSL. Boo is also a nice language to work with for day-to-day tasks. The SharpDevelop IDE project has support for Boo, so you get the usual benefits. There is an ongoing project to support Boo in Visual Studio, and I have an active interest in seeing support for that being established. I don't much care for which IDE, but I want a standard, checklist driven way of getting a language running.

Here is a secret, I tend to think and write in Boo, translating down to C# because I have to. Boo is a far more flexible language, and the ability to reprogram the compiler makes tedious tasks in other languages an amusing challenge. Those same tedious problems are usually overcome, to some degree, by throwing code generation at the problem until it goes away.image

In Boo, we would usually tell the compiler how we want to handle this, and let the compiler take over from there. This is similar to code generation, except that this is done at the compiler level, which means that we have far more knowledge about what is going on.

For example, let us say that we want to implement the singleton pattern. We need to do quite a bit of work to make it happen. Ensure that creating the singleton is thread safe, lazy initialization, make all the constructors private, ensure that serializable singleton remain singleton through serialization/deserialization, create static Instance property, have no static members on the singleton class.

To do this in most language we would have to follow an implementation pattern and write fairly repetitive code. In Boo, we can capture this usage pattern in an AstAttribute (more on that later) that will do all of that for us, at the cost of a single attribute declaration.

Similar capabilities allow correct usage of disposable, observable, and more advance techniques such as aspect oriented programming.

Some of the capabilities that Boo has can be had using IL Weaving tools such as PostSharp. The problem with those is that using them is something that happens after compilation. As a simple example, you would not be able to create a singleton in this manner, because you would not be able to generate the Instance property at the appropriate time.

Assuming that you want to use the singleton in the same assembly as it was defined, you will not be able to reference the Instance property, it will not exist until the post compile step has run. Boo, however, does this at compile time, so by the time the compiler get to the point where it is resolving the property, it has already been added.

In short, I really like the language and what it allows me to do.


[1] Well, if you are interested in creating a new language called Ba, you probably would want to make it clear where the project ancestry came from…

time to read 1 min | 183 words

Here is an interesting question, at what point you drive away so far from the original language that you lose the benefits of an internal DSL?

The following examples are several ways to represent the same thing, going from the extreme DSL to the no DSL to C#. I think that they all have value, and neither cross that line, but I believe that they will help make the point. The example here is a a simple MonoRail Controller:

#1:

controller Field:
    action Tag:
           @defaultTags = [ "green", "trees", "growth" ]

#2:

controller Field:
    def Tag():
         @defaultTags = [ "green", "trees", "growth" ]

#3:

controller Field:
    def Tag():
          PropertyBag["defaultTags"] = [ "green", "trees", "growth" ]

#4:

class FieldController(Controller):
   def Tag():
          PropertyBag["defaultTags"] = [ "green", "trees", "growth" ]

#5:

public class FieldController : Controller
{
	public void Tag()
	{
		PropertyBag["defaultTags"] = new string[] { "green", "trees", "growth" };
	}
}

Any comments?

Misconceptions

time to read 1 min | 58 words

I was explaining to a couple of team members about our stack (NHibernate, Castle, Boo, Rhino) and how the different pieces are hooked together.

When I got to talk to Boo, I explained that it is just like Brail, which we are using for the views. One of the guys then asked if Boo stands for Brail, Object Oriented

Mocking Boo

time to read 24 min | 4735 words

Okay, I built it to relax a bit, because I am extremely annoyed at the moment. I apologize in advance for the code quality, it is POC only, but still, I wouldn't generally release it like this.

What is this? Do you see the highlighted bit at the bottom? This is Boo code that invokes a Macro on compile. It will generate an adapter and an interface, so you don't have to do it manually. The implementation code is below:

image

class AdapterMacro(AbstractAstMacro):

      def Expand(macro as MacroStatement):

            if macro.Arguments.Count != 1 or not macro.Arguments[0] isa ReferenceExpression:

                  raise "adapter must be called with a single argument"

            entity = NameResolutionService.Resolve(macro.Arguments[0].ToString())

            raise "adapter only accept types" unless entity.EntityType == EntityType.Type

            BuildType(macro, entity)

           

      def GetModule(node as Node) as Boo.Lang.Compiler.Ast.Module:

            return node if node isa Boo.Lang.Compiler.Ast.Module

            return GetModule(node.ParentNode)

     

      def BuildType(macro as MacroStatement, type as IType):

            adapter = ClassDefinition(Name: "${type.Name}Adapter")

            adapter.Members.Extend(

                  Field(Name: "theTarget", Type: SimpleTypeReference(type.FullName) )

            )

           

            ctor = Constructor()

            ctor.Parameters.Add(ParameterDeclaration("target", SimpleTypeReference(type.FullName) ) )

            ctor.Body.Add(

                  BinaryExpression(BinaryOperatorType.Assign,

                        ReferenceExpression("theTarget"),

                        ReferenceExpression(Name: "target")

                  )

            )

            adapter.Members.Add(ctor)

            adapterInterface = InterfaceDefinition(Name: "I${type.Name}")

            GetModule(macro).Members.Add(adapter)

            GetModule(macro).Members.Add(adapterInterface)

            adapter.BaseTypes.Add( SimpleTypeReference(adapterInterface.FullName) )

            for member in type.GetMembers():

                  AddMethod(adapter, adapterInterface,  member) if member isa IMethod

           

            BooPrinterVisitor(System.Console.Out).Visit(adapterInterface)

            BooPrinterVisitor(System.Console.Out).Visit(adapter)

           

      def AddMethod(adapter as ClassDefinition,

            adapterInterface as InterfaceDefinition,

            method as IMethod):

           

            return unless method.IsPublic

           

            interfaceMethod = Method(Name: method.Name)

            forwarder = Method(Name: method.Name)

           

            if method.ReturnType.IsByRef or method.ReturnType.IsArray:

                  return

           

            args = []

            for param in method.GetParameters():

                 

                  if param.IsByRef or param.Type.IsArray:

                        return

 

                  forwarder.Parameters.Extend(

                        ParameterDeclaration(

                              Name: param.Name,

                              Type: SimpleTypeReference(param.Type.FullName)

                              )

                        )

                  interfaceMethod.Parameters.Extend(

                        ParameterDeclaration(

                              Name: param.Name,

                              Type: SimpleTypeReference(param.Type.FullName)

                              )

                        )

                  args.Add( ReferenceExpression(param.Name) )

                 

            adapterInterface.Members.Add(interfaceMethod)

            adapter.Members.Add(forwarder)

           

            mie = MethodInvocationExpression(

                  Target: AstUtil.CreateReferenceExpression("theTarget.${method.Name}")

                  )

            mie.Arguments.Extend(args)

            forwarder.ReturnType = SimpleTypeReference(method.ReturnType.FullName)

            interfaceMethod.ReturnType = SimpleTypeReference(method.ReturnType.FullName)

            if method.ReturnType == typeof(void):

                  forwarder.Body.Add(mie)

            else:

                  forwarder.Body.Add(ReturnStatement(mie))

Binsor 2.0

time to read 4 min | 671 words

All credit should go to Craig Neuwirt, for some amazing feats of syntax. He has managed to extend Binsor to include practically all of Windsor's configuration schema. This is important because previously we had to resort to either manual / ugly stuff or go back to XML (see: manual & ugly).

This is important because some of the more interesting things that you can do with Windsor are done using the facilities, and Craig has made sure that Binsor will support the main ones in a natural manner, including out of the box support for the standard configuration model.

Let us start by using the standard configuration model for a moment, the Active Record Integration Facility is expecting to be configured from XML, but we can configure it like this:

facility arintegration, ActiveRecordFacility:
	configuration:
		@isWeb = true, isDebug = true
		assemblies = [ Assembly.Load("My.Assembly") ]
		config(keyvalues, item: add):
			show_sql = true
			command_timeout = 5000
connection.isolation = 'ReadCommitted' cache.use_query_cache = false dialect = 'NHibernate.Dialect.MsSql2005Dialect' connection.provider = 'NHibernate.Connection.DriverConnectionProvider' connection.driver_class = 'NHibernate.Driver.SqlClientDriver' connection.connection_string_name = 'MyDatabase' end end

To compare, here is the equivalent  XML configuration.

<facility id="arintegration" 
	type="Castle.Facilities.ActiveRecordIntegration.ActiveRecordFacility, Castle.Facilities.ActiveRecordIntegration" 
	isWeb="true" 
	createSchema="true">
 <assemblies>
	<item>My.Assembly</item>
 </assemblies>
 <config>
	<add key="hibernate.cache.use_query_cache" value="true" />
	<add key="hibernate.connection.isolation" value="ReadCommitted" />
	<add key="hibernate.show_sql" value="false" />
	<add key="hibernate.dialect" value="NHibernate.Dialect.MsSql2005Dialect" />
	<add key="hibernate.connection.driver_class" value="NHibernate.Driver.SqlClientDriver" />
	<add key="hibernate.connection.connection_string_name" value="MyDatabase" />
	<add key="hibernate.connection.provider" value="NHibernate.Connection.DriverConnectionProvider" />
 </config>
</facility>

We don't have any significant reduction in the number of lines. This is mostly because we need to specify a lot of items for the facility to work, but we do have significant reduction in the noise that we have to deal with.

But, frankly, this isn't the best sample, let us take a look at the shortcuts that Binsor now provides, shall we?

Event Wiring

XML configuration:

<component 
	id="SimpleListener" 
      type="Castle.Facilities.EventWiring.Tests.Model.SimpleListener, Castle.Facilities.EventWiring.Tests" />

<component 
	id="SimplePublisher" 
      type="Castle.Facilities.EventWiring.Tests.Model.SimplePublisher, Castle.Facilities.EventWiring.Tests" >
      <subscribers>
      	<subscriber id="SimpleListener" event="Event" handler="OnPublish"/>
	</subscribers>
</component>

And the Binsor configuration to match it:

component simple_listener, SimpleListener

component simple_publisher, SimplePublisher:
	wireEvent Publish:
		to @simple_listener.OnPublish

Now that is far clearer, isn't it?

Factory Support

XML configuration:

<component id="mycompfactory" 
      type="Company.Components.MyCompFactory, Company.Components"/>

<component id="mycomp" 
	type="Company.Components.MyComp, Company.Components"
	factoryId="mycompfactory" 
	factoryCreate="Create" />

Binsor configuration:

component mycompfactory, MyCompFactory

component mycomp, MyComp:
	createUsing @mycompfactory.Create

Other new features:

  • Integrating Castle Resources, so now we can use Binsor over several files, which can be located anywhere, including assembly resources, for instance.
  • Easy support for startable and life styles.
  • Extension points for future needs.
time to read 2 min | 391 words

You were just handed a strange DSL implementation, it does stuff, and it may be cool, but you have no idea how it works.

Craig Neuwirt recently did a major overhaul of Windsor ( I am going to post soon with the details of how cool it is now ), but I am now faced with the question, how do you grok such a thing? I thought that it would be useful to put a list of what I am doing to understand how the internals now works.

It goes without saying, but the nitpickers will ask, that a DSL implementation is code like any other, and as such, it can have good & bad implementations. I think that what Craig has done was amazing. It makes for an interesting reading, I actually had to take notes, to make sure that I will not miss any of the cool stuff.

  • Understand the domain.
    This is critical, because if the DSL is chicken scratching to you, you won't be able to get the implementation.
  • Get the tests.
    First things first, make sure that you have tests that ensure that you can understand what the DSL is supposed to do. In addition to that, it allows you to debug a simplified scenario repeatedly, so you can walk through that and figure out what is going on.
  • Take the simplest scenario and run it, then open the resulting DLL in Reflector
    This is important, because you need to be able to see what the differences are between the DSL and the actual executed code are. This allows you to understand in a deep level what the various elements of the language do.
  • Identify a transformation, and follow how it is being built throughout the code
    It will allow you to understand the concepts and patterns used through the code.
  • Follow each transformation throughout its own code path only
    Fairly often you can get side tracked by trying to put the entire program in your head, this is usually not possible, track a single path through the code to completion. After that, you can start tracking other paths, but it is important to have an understanding of the full cycle, and then expand from that.

Hm, not really that different from how I would approach an unfamiliar code base, come to think about it.

FUTURE POSTS

No future posts left, oh my!

RECENT SERIES

  1. Production postmorterm (2):
    11 Jun 2025 - The rookie server's untimely promotion
  2. Webinar (7):
    05 Jun 2025 - Think inside the database
  3. Recording (16):
    29 May 2025 - RavenDB's Upcoming Optimizations Deep Dive
  4. RavenDB News (2):
    02 May 2025 - May 2025
  5. Production Postmortem (52):
    07 Apr 2025 - The race condition in the interlock
View all series

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats
}