CodeGen

We'll just code-gen our way out of here

Ayende Rahien — Sun, 16 Dec 2007 08:54:16 GMT

First, we have the implementation of the SendPropertyChanged method, can you spot the bug?

protected virtual void SendPropertyChanged(String propertyName)
{
	if ((this.PropertyChanged != null))
	{
		this.PropertyChanged(this, new PropertyChangedEventArgs(propertyName));
	}
}

Then, you have:

public string CustomerID
{
	get
	{
		return this._CustomerID;
	}
	set
	{
		if ((this._CustomerID != value))
		{
			this.OnCustomerIDChanging(value);
			this.SendPropertyChanging();
			this._CustomerID = value;
			this.SendPropertyChanged("CustomerID");
			this.OnCustomerIDChanged();
		}
	}
}

And this:

public Customer Customer
{
	get
	{
		return this._Customer.Entity;
	}
	set
	{
		Customer previousValue = this._Customer.Entity;
		if (((previousValue != value) 
					|| (this._Customer.HasLoadedOrAssignedValue == false)))
		{
			this.SendPropertyChanging();
			if ((previousValue != null))
			{
				this._Customer.Entity = null;
				previousValue.Orders.Remove(this);
			}
			this._Customer.Entity = value;
			if ((value != null))
			{
				value.Orders.Add(this);
				this._CustomerID = value.CustomerID;
			}
			else
			{
				this._CustomerID = default(string);
			}
			this.SendPropertyChanged("Customer");
		}
	}
}

Both properties show exactly why I despise code generation as an architectural approach.

Instead of taking the appropriate route, and actually solving the problem at hand, they just threw code at it until it looked like it worked.

RunSharp: Reflection Emit for Mere Mortals

Ayende Rahien — Sat, 03 Nov 2007 03:50:21 GMT

I have been dealing with IL generation using Reflection Emit for about two years. I believe that I have enough experience with it to get a feeling for how it goes, and it usually goes slowly and painfully. There are wrappers around it (Castle Dynamic Proxy has a AST for this), but they are fairly specialized, and anyone somehow I always find myself needing to fix the code that build the AST, mostly because I have users that throws weird curve balls at Rhino Mocks.

Nevertheless, IL generation is a powerful technique, just very cumbersome to deal with. RunSharp is a OSS project that aims to solve this issue, it generate code on the fly, using high-level syntax.

This means that you can write code like this:

g.Assign(x, o.Invoke("MyMethod", arg1, arg2));
g.Assign(y, o.Property("MyProperty"));
g.Assign(o.Property("MyProperty"), z);
g.Assign(f, o.Field("myField"));

And it will generate the appropriate:

x = o.MyMethod(arg1, arg2);
y = o.MyProperty;
o.MyProperty = z;
f = o.myField;

I am impressed, if this works, it can mean some very interesting possibilities. I took a very brief glance at the code, and it is using the ILGenerator from the framework, I would be very interested in getting this to work on (a) dynamic methods, (b) cecil.

The license is GPL, however, which means that it is a problem to use in any scenario. In my opinion, such a thing should be LGPL, which would allow its use in other projects, but that is a subject for another day.

Thanks Roy, for finding it.

Dynamic Methods

Ayende Rahien — Mon, 29 Oct 2007 20:07:48 GMT

I don't hear it talked about, but the CLR has a very efficient way to generate code at runtime. Probably this is because this code generation stuff is something that is accessible through IL generation only, and that is not for the faint of heart. Nevertheless, there are some very useful uses for this. NHibernate is utilizing this approach to avoid the costs of reflection, for instance.

Let us take a look about a simple scenario, we want to translate any delegate type with two parameters to a call to an instance method on our class:

public class Program
{
	private static void Main(string[] args)
	{
		new Program().Execute();
	}

	private void Execute()
	{
		//instance that has events that we want to subscribe the adapter to 
		DataGridView dataGridView1 = new DataGridView();
		EventInfo ei = dataGridView1.GetType().GetEvent("RowPrePaint");

		ParameterInfo[]pia = ei.EventHandlerType.GetMethod("Invoke").GetParameters();

		MethodInfo methodInfo = this.GetType().GetMethod("Handler", 
			new Type[]{typeof (object), typeof (object)});

		DynamicMethod mtd = new DynamicMethod(
			"Adapter",
			typeof(void),
			new Type[]
				{
					typeof (Program), // this 
					pia[0].ParameterType,// sender
					pia[1].ParameterType // e
				}, this.GetType(), true);

		ILGenerator gtr = mtd.GetILGenerator();
		gtr.Emit(OpCodes.Ldarg_0); // this
		gtr.Emit(OpCodes.Ldarg_1); // sender
		gtr.Emit(OpCodes.Ldarg_2); // e
		gtr.Emit(OpCodes.Call, methodInfo);
		gtr.Emit(OpCodes.Ret);

		// generate a delegate bound to this object instance
		Delegate dynamicDelegate = mtd.CreateDelegate(typeof(DataGridViewRowPrePaintEventHandler), this);
		//register the adapter
		ei.AddEventHandler(dataGridView1, dynamicDelegate);


		dataGridView1.GetType().GetMethod("OnRowPrePaint", BindingFlags.NonPublic | BindingFlags.Instance)
			.Invoke(dataGridView1, new object[] { null });
	}

	// method that handles the call
	public void Handler(object x, object y)
	{
		Console.WriteLine("{0}: {1}, {2}", this.GetHashCode(), x, y);
	}
}

Take into account that you are probably going to want to cache the method anyway, but this is a cool, if long winded way of achieving this. Personally, in this scenario I would probably simply write a reflection based wrapper, the complexity doesn't really have justification in such a case, but this is just an example, of course.

Working with ANTLR: HQL Grammar

Ayende Rahien — Sat, 11 Aug 2007 03:54:20 GMT

I am currently trying to build an ANTLR grammar for HQL. There is already an existing one for Hibernate 3, but that one is based on ANLTR 2.x and supports quite a bit more than NHibernate does at the moment (DML statements, for instance). After several failed attempts to port the grammar to ANTLR 3 and generate C# code out of it, I gave up and started building my own.

I have read the ANTLR book not that long ago, so I ought to have a known what was in store for me. I didn't. I found out that this require a totally different mode of thinking. My recursion muscle is very tired at the moment, but I managed to create a simple grammar for:

select x.y, z.b from Entity as e join e.Children as c

I kept having false starts with the thing, until I went and read Boo's source, and figure out how Boo's parser works. Basically, instead of letting the tool to generate the parser and work of the generated tree, this approach calls for constructing our own tree while we parse the source.

As you can see, we constantly pass the Query instance down to lower rules, so we can operate on it and build our tree. This is much easier than trying to handle the CommonTreeAdaptor [sic] and derivatives.

I want to make the parser and the resulting AST as smart as possible, before trying to plug it into NHibernate's itself. That is going to be a significant undertaking, and I would like to have help, so feel free to contribute.

Now that I have the initial stuff going, I am going to refactor it a bit to match this BNF (http://www.hibernate.org/89.html), and yes, I know it is outdated.

There are about a dozen tests for the syntax yet, so it is possible to just grab it and start working on it.

You can grab the source from: https://rhino-tools.svn.sourceforge.net/svnroot/rhino-tools/experiments/Hql

Observations on Learning

Ayende Rahien — Sat, 30 Jun 2007 20:11:42 GMT

Here is an observation on learning. When I was at high school, I was thought Pascal, and I couldn't for the life of me understand dynamic memory allocation. I had little problem with everything else, but dynamic memory allocation (better known as pointers) was a mystery wrapped in an enigma stashed inside a headache.

About a year later, I was learning C++, and was one of the first in the class that grasped pointers and their usages. I remember trying to explain ***pppHead (sparse matrix) to another student, and he drew blank after the first level of indirection. I don't think that the quality of the teachers was that different, and the material is basically the same, but I grokked the second and couldn't figure out the first.

I have run into this many times since, usually a piece of technology just doesn't make sense to me, and at one point, it clicks together, and it is "Oh, that is simple!"

For a while now, I have been feeling my lack of knowledge in the area of parsers, and I kept trying to learn ANTLR on my own. I got to the point where I could read EBNF fairly well, and actually make sense of it, but taking the next step to actually building a language has been beyond me. Yesterday I picked up The Definitive ANTLR Reference, and I have been going through it with a fairly rapid pace. I don't think that at my level, the book is offering something that isn't already available online, but I have been able to understand how things mesh together much better now.

I feel that now, I am not competent with parser building, it is certainly something that I can be with a reasonable amount of real world practice. In other words, I think that I am going to be able parsers and parser building to my toolbox.

What kind of persistence would Ayende write without OR/M?

Ayende Rahien — Wed, 06 Jun 2007 18:56:56 GMT

Rodrigo thinks that I am ignoring the obvious when I think about persistence only in terms of RDBMS. He is probably right.

I want to point out that the last time I wanted to have a non-DB persistence, I used C# as the file format. That was fairly successful, although I will probably won't do it again.

NHibernate Query Generator - More Collections Magic

Ayende Rahien — Tue, 05 Jun 2007 20:53:06 GMT

I wouldn't have expected it to be this hard*, but it is alive!

Here is the query:

   1:  User one = User.FindOne(

   2:         Where.User.Blogs

   3:              .With(JoinType.InnerJoin)

   4:                   .Name == "Ayende @ Blog"

   5:  );

And the generated SQL:

   1:  SELECT

   2:     this_.Id as Id4_1_,

   3:     this_.Name as Name4_1_,

   4:     this_.Email as Email4_1_,

   5:     blog1_.Id as Id3_0_,

   6:     blog1_.Name as Name3_0_,

   7:     blog1_.Author as Author3_0_

   8:  FROM Users this_ inner join Blogs blog1_

   9:     on this_.Id=blog1_.Author

  10:  WHERE blog1_.Name = @p0

And, because Rob has asked, here is how you do a more complex query, over Many To Many association:

   1:  User one = User.FindOne(

   2:      Where.User.Name == "Ayende" &&

   3:      Where.User.Roles.With().Name == "Administrator"

   4:  );

And the generated SQL:

   1:  SELECT this_.Id as Id0_1_,

   2:     this_.Name as Name0_1_,

   3:     this_.Email as Email0_1_,

   4:     roles3_.UserId as UserId__,

   5:     role1_.Id as RoleId,

   6:     role1_.Id as Id9_0_,

   7:     role1_.Name as Name9_0_

   8:  FROM Users this_ inner join UsersRoles roles3_

   9:     on this_.Id=roles3_.UserId

  10:        inner join Roles role1_

  11:           on roles3_.RoleId=role1_.Id

  12:  WHERE this_.Name = @p0 and role1_.Name = @p1

It is in the repository now, but right now I would consider it beta stage, it works, but the generation will probably die if you have any sort of interesting schemas (specifically, if you have collections of value types, it is supposed to die horribly with confusing error message).

I intend to do a screen cast about this area of querying, searching and persistence soon.

* Right now code generation competes with template magic and three stars code in my internal dislike list. Trying to get the sort of syntax that you see about is really stretching C# to its limits. When can I get a Method Missing implementation on C#? That would actually make my life easier?

On the advantages of one off tools

Ayende Rahien — Mon, 04 Jun 2007 22:54:14 GMT

I don't like general purpose code generation, but I am fan of special purpose ones. The main reason that I don't like the general purpose ones is in my experience they have invariably fell into the "Generate code that I don't like" or "Requires too much futzing to get to work the way I want it." Part of it is related to the fact that I consider generated code as a maintainability concern as well, and I am very nervous about trying to do that.

Nevertheless, I do like code generation, I tend to write simple code-generators for a lot of purposes. In my current project, we have a lot of churn in database schema wise (add field, rebuild, get a new DB), but we also have all kinds of ETL process to and from the DB, so that makes life more interesting. I wrote a code generator based on SMO that reads the DB schema after NHibernate generates it, and basically handles ~85% of the ETL process mess.

The nice thing about doing a one off tool is that I can make a lot of assumptions about the structure that I will be using (all tables has PK named Id, for instances) that a general purpose tool cannot make.

Dynamic Language Runtime on top of the CLR: This Is BIG

Ayende Rahien — Mon, 30 Apr 2007 20:33:32 GMT

Check this out. Even though that Jim says that it is on the IronPython site, I can't find it, but I am still loving it. Making the CLR more friendly to dynamic languages is a Good Thing in general, but the thing that excites me is the possiblity that it can be leveraged from my code as well.

I paid my IL taxes, and while Linq expressions are nice, I would like to get solid support for runtime code generation without having to do it in the assembly level.

Quick & Dirty CodeGen

Ayende Rahien — Sun, 15 Apr 2007 06:11:23 GMT

I needed to get some code that would map an XML file to a database table. Not being particularily fond of doing it by hand, I whipped out this statement:

select '

if node.SelectSingleNode("' + column_name + '/text()") is not null:

row["' + column_name + '"] = node.SelectSingleNode("' + column_name + '/text()").Value

else:

row["' + column_name + '"] = DBNull.Value'

from information_schema.columns

where table_name = 'Content'

I am doing about 60% of my code gen with SQL and Regex, I think.