Saturday, September 06, 2008
#
Getting Things Done, On Time
Today I implemented refactoring support for a DSL.
Basically, it is Extract Business Condition, and it was explicitly modeled after the way R# handles Extract Variable. It even share the same shortcut, ctrl+alt+v.
I also took a stub in implementing automatic pattern recognition, so when the system recognize a common usage pattern, it will automatically refactor it to a high level abstraction. It works, although I think that I can make it even more flexible than it is now.
Now, if someone from the Resharper team is actually reading this, they would know that I am lying. There is no way of doing something of this magnitude in just one day, not even if you have an extremely helpful compiler.
And they would be right. I didn't try to tackle a feature of this magnitude. What I did do was to find the most common scenario for this feature and nail that.
I am taking this approach explicitly and deliberately. With the end result that I get to show value very rapidly. And yes, the customer is made aware of the limitations of this approach. I also tell them that they can get the feature by tomorrow, with error message if they are trying to do something that is not supported.
Trying to support 100% is hard, trying to support just 20% turn out to be (not quite) easy.
And now you get to nitpick it to death, I won't respond for about a day, since I am just about to board a flight.
Friday, September 05, 2008
#
Choose a workshop
I am going to give a workshop or two at the ALT.Net Austin in the end of October. Those will be free (as in beer) and will be recorded & available on the net afterward. Right now I want to do on on writing DSLs, but I have another which is basically blank at the moment. I have too many subjects that I can talk about, and too many levels at which I can talk about them.
So, this is your chance to help me. If you are going to be there, what would you like to have a workshop about?
And no, a question like NHibernate is not acceptable, it is too broad. Are we talking about NHibernate best practices, high scalability, tips and tricks or advance usages. I can do a three hours workshop on any of them.
Suggestions?
Thursday, September 04, 2008
#
Answer: Don't stop with the first DSL abstraction
The problem as it was stated was of rules that looked like this:
upon bounced_check or refused_credit:
if customer.TotalPurchases > 10000: # preferred
ask_authorization_for_more_credit
else:
call_the cops
upon new_order:
if customer.TotalPurchases > 10000: # preferred
apply_discount 5.precent
upon order_shipped:
send_marketing_stuff unless customer.RequestedNoSpam
I don't like it, and the reason isn't just that we can introduce IsPreferred.
I don't like it because the abstraction facilities here are poor. We have basically introduced events and business rules, maybe with a sprinkling of a domain model, but nothing really meaningful. Such system will die under their own weight in any situation of significant complexity (in other words, in all real world situations).
Let us consider the problem in reverse, shall we? We have various conditions and actions upon which we can act. But the logic is scattered all over the place, making it hard to read, modify, understand and work with. When such a system compose of the lifeblood of the business, the business usually adapts, and starts to talk in the terms of the system. However, they tend to lose the ability to think about things in way that would be more meaningful.
I listened today to a business person trying to explain some concept that he wanted to make. It took him several tries to explain the business problem because he was focused on the technical one. The system has a corrupting affect on it. I call this the Babel Syndrome, the reverse of DDD's ubiquitous language.
Let us see if we can get a high level of meaning out of the above DSL, shall we? First, we restate our problem, instead of dealing with events and conditions for responding the events, we deal with business responses for scenarios. It doesn't sound like much of a difference, but in actuality, there is a big difference between the two.
The most important of those differences is the change from handling the events to handling a business scenario in a given context. In other words, instead of asking what we should do when a check is bounced, we need to ask a totally different question. "When the customer is preferred, what should the response be for bounced check?"
This is anything but a minor change in the the way we think about the language and how we operate on it. Let us see the DSL script, after which we can discuss how it affects us. These are the contents of the default.boo file:
upon order_shipped:
send_marketing_stuff unless customer.RequestedNoSpam
upon bounced_check or refused_credit:
call_the cops
This will be executed for all orders, like before. Now, let us look at preferred_customer.boo, and what concepts it express.
when customer.TotalPurchases > 10000 # preferred
upon new_order:
apply_discount 5.precents
upon bounced_check or refused_credit:
ask_authorization_for_more_credit
And now we are getting to see some of the more interesting parts of the difference. We are now talking in terms of a business scenario. When we have a preferred customer, and something happen, how should we respond?
This change is a well known refactoring: conditional to polymorphism. In other words, we just created the strategy pattern with a DSL. The difference here is that the script have an active role in deciding whatever it can deal with the scenario or not (in other words, chain of responsibility, and the pattern I am going to mention).
When we need to handle some business scenario, we are going to execute all the scripts, with the default.boo being the last one to run. If any of the scripts accepted the scenario as valid and has specific action to take, it has the option to do so.
Enough about the implementation, let us go back to the concepts. We can make now talk to the business people in a way that is far more concise and natural. Instead of having to focus on all permutations of a possible event, we can now talking about a specific scenario and how we handle the business event in that context. Not only is this more readable, it is easier by far to actually define such things as what is the meaning of a preferred customer. I can open the DSL and actually read it.
Similar approaches are very useful when you recognize that the code is asking to be given a more explicit shape than just generic rules. Don't let your DSL be whatever you started with. Find and actively extract higher level meanings whenever it is possible.
A deeper examination of this DSL, how to build and use it is likely to compose most of chapter 13, as a real world example of a complex DSL. Who do you think?
Given this approach, how would you design an offer management DSL?
Challenge: Don't stop with the first DSL abstraction
I was having a discussion today about the way business rules are implemented. And large part of the discussion was focused on trying to get a specific behavior in a specific circumstance. As usual, I am going to use a totally different example, which might not be as brutal in its focus as the real one.
We have a set of business rules that relate to what is going to happen to a customer in certain situations. For example, we might have the following:
upon bounced_check or refused_credit:
if customer.TotalPurchases > 10000: # preferred
ask_authorizatin_for_more_credit
else:
call_the cops
upon new_order:
if customer.TotalPurchases > 10000: # preferred
apply_discount 5.precent
upon order_shipped:
send_marketing_stuff unless customer.RequestedNoSpam
What is this code crying for? Here is a hint, it is not the introduction of IsPreferred, although that would be welcome.
I am interested in hearing what you will have to say in this matter.
And as a total non sequitur, cockroaches at Starbucks, yuck.
Persistent DSL Caching
This is a note to myself, because I don't have the time for a proper post. When you are dealing with a DSL that contains more than just a few scripts, you really being to care about compilation times. Even with caching, this can be a problem.
The solution is the same that we have been using for the last three to four decades, don't compile if the source hasn't changed.
The code to make this happen using Rhino DSL is here:
public override CompilerContext Compile(string[] urls)
{
var outputAssemblyName = OutputAssemblyName(urls);
if (CanUseCachedVersion(outputAssemblyName, urls))
return new CompilerContext { GeneratedAssembly = Assembly.Load(File.ReadAllBytes(outputAssemblyName)) };
return base.Compile(urls);
}
private bool CanUseCachedVersion(string outputAssemblyName, string[] urls)
{
var asm = new FileInfo(outputAssemblyName);
if(asm.Exists==false)
return false;
foreach (var url in urls)
{
if(File.GetLastWriteTime(url) > asm.LastWriteTime)
return false;
}
return true;
}
And in the CustomizeCompiler method:
protected override void CustomizeCompiler(BooCompiler compiler, CompilerPipeline pipeline, string[] urls)
{
compiler.Parameters.OutputAssembly = OutputAssemblyName(urls);
// add implicit base class here...
if (pipeline.Find(typeof(SaveAssembly)) == -1)
pipeline.Add(new SaveAssembly());
}
It is impossible to overstate how big a difference this can make.
Wednesday, September 03, 2008
#
Chapter 11 - Documenting your DSL
I am going to start writing this soon. Any suggestions?
As a bait, I am going to apply the techniques that I intend to describe to Binsor, which should give you incentive to say what you really want from a DSL documentation... :-)
Chapter 10 is done, happy
A few hours ago I finished writing Chapter 10. Looking at the time on the calendar, it didn't take too long, about a month. Looking at it from the point of view of effort involved, I feel about as tired as if it took three years.
Other chapters took longer, but didn't take out quite as much effort. Not sure what this means... but I am happy I am done with it (for a given value of done, I will still need to review and read it at least three dozen times).
A balancing act
Probably one of the hardest challenges that I am facing with writing the book is to know what to say and what to leave unsaid.
Phrasing it another way, it is choosing at what level to talk to the reader. On the one hand, I really want the reader to be able to make immediate use of the concepts that I am talking about, which drive me to do more practical demonstrations, code samples and covering more common situations. On the other hand, those take up a lot of room, and they tend to be boring if you don't need exactly what you need right this moment.
High level concepts, open ended possibilities and assuming a bit about the reader knowledge level makes for a book that is much more narrowly focused, and I think that it more valuable. However, it also tend to leave readers unsatisfied, because not everything is explained.
Currently I am writing a UI focused chapter, and to get a good experience from the UI you need to invest a lot of time. Metric tons of it. I am trying to chart the way and show how this can be done, but without getting mired in all the actual minute details.
This is a tough balancing act, and I am not sure if I am succeeding.
Is this valuable?
Going back to the theme of visually editing a DSL, we have something like this:
The backend to this cutey is pattern recognition with custom UI on top. You have to teach it about each pattern that you use with it, but you can get pretty smart with how it works while using a fairly brute force (and simple) techniques.
Thoughts?
Nullifying Null
One of the more annoying problems with building rules that are also code is that you have to deal with code related issues. One of the more common ones is NullReferenceException.
For example, let us say that we have the following rule:
when Order.Amount > 10 and Customer.IsPreferred:
ApplyDiscount 5.precent
We also support a mode in which a customer can create an order without actually registering on the site (anonymous checkout).
In this scenario, the Customer property is null. We can rewrite the rule to look like this:
when Order.Amount > 10 and Customer is not null and Customer.IsPreferred:
ApplyDiscount 5.precent
But I think that this is extremely ugly. We can also decide to return a default instance of the customer when it is not there, but here I want to show you another way to handle this. We define the rule as invalid when Customer is not there, so it should not be run. The question is how we can know that.
The dirty way is to do something like this:
var referencesCustomer = File.ReadAllText(ruleName).Contains("Customer");
if(referencesCustomer && Customer == null)
return;
If you gagged when seeing this code, that is a good sign. Let us solve this properly. First, we want some help from the compiler, so let us inspect the when() meta method that we have seen in the previous post a little closer.
[Meta]
public static ExpressionStatement when(Expression condition, BlockExpression action)
{
var ctor = new BlockExpression();
var conditionFunc = new Block();
conditionFunc.Add(new ReturnStatement(condition));
ctor.Body.Add(
new BinaryExpression(
BinaryOperatorType.Assign,
new ReferenceExpression("Condition"),
new BlockExpression(conditionFunc)
)
);
Expression serialize = new CodeSerializer().Serialize(condition);
Builder.Revisit(serialize);
ctor.Body.Add(
new BinaryExpression(
BinaryOperatorType.Assign,
new ReferenceExpression("ConditionExpression"),
serialize
)
);
ctor.Body.Add(
new BinaryExpression(
BinaryOperatorType.Assign,
new ReferenceExpression("Action"),
action
)
);
return new ExpressionStatement(
new MethodInvocationExpression(
ctor
));
}
We take the cal to the when method and transform it to the following code:
delegate
{
Condition = () => Order.Amount > 10 && Customer.IsPreferred;
ConditionExpression = (Expression<Func<bool>>)() => Order.Amount > 10 && Customer.IsPreferred;
Action = delegate
{
// not interesting for this post
};
}();
I am translating to C# 3.0 here in order to make it easier to grasp the concept. The real code is in Boo, of course, and is more interesting. The most fascinating concept here is the use of CodeSerializer, which will turn the condition that we passed into an AST that we can access at runtime. I tried to simulate that by doing an explicit cast to expression tree, which would give similar result in C#).
Having the AST of the code at runtime, even if we don't want to change it (a totally different concept) is incredibly powerful. In this case, we are going to use this to detect when we are referencing a null property and marking the rule as invalid.
Here is the code:
public void Evaluate()
{
var references = new List<string>();
new InlineVisitor
{
OnReferenceExpression = r => references.Add(r.Name);
}.Visit(ConditionExpression);
if(references.Contains("Customer") && Customer == null)
return;// rule invalid
if(Condition())
Action();
}
This is a very simple example of how you can add smarts to the way that your code behaves. This technique is the foundation for a whole host of options. I am using similar approaches for adaptive rules and for auditable actions. Fun stuff, if I say so myself.
Tuesday, September 02, 2008
#
How to execute a set of statements in an expression
One of the most common problems with using Boo's Meta Methods is that they can only return expressions. This is not good if what you want to return from the meta method is a set of statements to be executed.
The most common reason for that is to initialize a set of variables. Obviously, you can call a method that will do it for you, but there is a simpler way.
Method invocation is an expression, and anonymous delegate definition is also an expression. What does this tells you? That this is an expression as well:
// the whole thing is a single expression.
delegate (int x)
{
Console.WriteLine(x);
Console.WriteLine("another statement");
}(5);
You generally won't use something like that in real code, but when you are working with the AST directly, expressions vs. statements require a wholly different world view.
Anyway, let us see how we can implement this using Boo.
[Meta]
public static ExpressionStatement when(Expression condition, BlockExpression action)
{
var func = new BlockExpression();
var conditionFunc = new Block();
conditionFunc.Add(new ReturnStatement(condition));
func .Body.Add(
new BinaryExpression(
BinaryOperatorType.Assign,
new ReferenceExpression("Condition"),
new BlockExpression(conditionFunc)
)
);
Expression serialize = new CodeSerializer().Serialize(condition);
RuleBuilder.Revisit(serialize);
func .Body.Add(
new BinaryExpression(
BinaryOperatorType.Assign,
new ReferenceExpression("ConditionExpression"),
serialize
)
);
func .Body.Add(
new BinaryExpression(
BinaryOperatorType.Assign,
new ReferenceExpression("Action"),
action
)
);
return new MethodInvocationExpression(func);
}
This trick lets you use the meta method to return several statements, which allows to do several property assignments (something that you generally cannot do in a single expression). I'll go over the actual meaning of the code (rather than the mechanics) in a future post.
Inline Anonymous Visitors
One of the most common chores when working with compilers is the need to create special visitors. I mean, I just need to get a list of all the variables in the code, but I need to create a visitor class and execute it in order to get the information out. This is not hard, the code is something like this:
public class ReferenceVisitor : DepthFirstVisitor
{
public List<string> References = new List<string>();
public void OnReferenceExpression(ReferenceExpression re)
{
References.Add(re.Name);
}
}
public bool IsCallingEmployeeProperty(Expression condition)
{
var visitor = new ReferenceVisitor();
visitor.Visit(condition);
return visitor.References.Contains("Employee");
}
Doing this is just annoying. Especially when you have to create several of those, and they make no sense outside of their call site. In many ways, they are to compilers what event handlers are to UI components.
What would happen if we could create a special visitor inline, without going through the "create whole new type" crap? I think that this would be as valuable as anonymous delegates and lambdas turned out to be. With that in mind, let us see if I can make this work, shall we?
public bool IsCallingEmployeeProperty(Expression condition)
{
var references = new List<string>();
new InlineVisitor
{
OnRefefenceExpression = re => references.Add(re.Name)
}.Visit(condition);
return references.Contains("Employee");
}
Especially in the more moderately complex scenarios, such a thing is extremely useful.
The implications of Google Chrome
So, Google is coming out with a new browsers, while at the same time they are also responsible for a large part of both FireFox and Opera's budgets.
I think that this is a very interesting development. In particular, because Google thinks about the browser as a complementary offer to what it does, or as a baseline platform, not as the actual end result.
This get really interesting when you think that in this scenario, Google can leverage what are effectively Killer Applications in order to migrate people from one browser to another. If YouTube worked better with Chrome than with IE, I think it would be a very powerful motivator to move. And Google has dozens of such high value assets.
I am pretty sure that Google will produce plugins for anything new it creates, so other browsers can work with it as well (to do otherwise is to risk monopoly charges), but if used properly, it will allow Google to leverage its own power to produce its own de facto standards, which browsers will have to follow.
The focus on creating a browser which is focused primarily on allowing application development and hosting (vs. mere browsing) is likely to aid in setting the new baseline standard of what a browser platform should provide for the web applications that are hosted in it.
From my point of view, I think that this is going to allow Google to take a much more active role in shaping the environment in which their applications are living. From that perspective, it seems like a very natural step for them.
Review: Hibernate Search in Action
I just finished reading Hibernate Search in Action, and I loved it. I should point out that I was the porter of Hibernate Search to NHibernate Search, so I had some previous expertise in the topic. In addition to that, I approached this book at an angle completely orthogonal to the expected audience. Unlike most "in Action" books, I did not intend to make immediate use of the code and approaches suggested in the book. Instead, I looked to the book as a way to deepen my understanding of the tool and how it works.
I am impressed, massively so, that it did so well in this regard for someone who has gone through the entire source code of the project several times.
I'll not bore you with the actual details, you can get the actual content summary of off the site. From my perspective, after reading this book I know that I am going to take a completely different approach for most complex search scenarios, and I think that I have the practical theoretical knowledge to deal with it.
I highly recommend the book if you actually need to deal with Hibernate Search, but I would recommend it to people who are not using it, because it contains some important eye opening concepts if you are not used to full text search tools capabilities. As a nice bonus, I was able to take the information in the book and use it to discuss a problem the customer was having, ending up with something that I consider far superior of the solution that they currently employ.
It is not out yet, and I reviewed a non final copy, but you can order the PDF right now, and just reading the freely available first chapter is valuable in itself.
Sunday, August 31, 2008
#
Hug a developer
This is something that has been making the rounds lately. Just about any developer that have seen it was greatly amused.
Check out the video.
This is so good, I intend to make this into a part of several of my presentations.
Saturday, August 30, 2008
#
Does you application has a blog?
I was having dinner with Dru Sellers and Evan Hoff and Dru brought something up that really sparked my imagination. To put it in more concrete words, what Dru said started a train of thought that ended up with another mandatory requirement for any non trivial application.
Your application should have a blog.
Now, pay attention. I am not saying that the application team should have a blog, I am saying that the application should have one. What do I mean by that? As part of the deployment requirements for the application, we are going to setup a blog application that is an integrated component of the application.
Huh? I am building Order Management application, why the hell do I need to have a blog as part of the application? Yes, PR is important, and blogs can get good PR, but what are you talking about?
This is an internal blog, visible only for the internal users, and into it the application is going to blog about interesting events that happened. For example, starting up the application would also cause it to post to its blog about that, which can look like this:
This looks like log messages that were written by a PR guy. What is the whole point here? Isn't this just standard logging?
Not quite. This is an expansion of the idea of system alert, where the system can proactively determine and warn about various conditions. This idea is anything but new, you are probably familiar with the term the Operation Database. But this approach has one key factor that is different.
Social Engineering
Using a blog, and using this style of writing, making it extremely clear what should and should not go there as a message. You obviously are not going to want to treat this as a standard log, where you just dump stuff in. From the point of view of actually getting this through, this make a task that is often very hard into a very simple "show by example".
From the point of view of the system as a whole, now business users have a way to watch what the system is doing, check on status updates, etc. More than that, you can now use this as a way to post reports (weekly summary, for example) and in general vastly increase the visibility of the system.
Using RSS allows syndication, which in turn also also easy monitoring by an admin, without any real complexity getting in the way. For that matter, you can get the business user to subscribe to it with Outlook (if they don't already have a standard RSS reader) and get them on board as well.
Now, this is explicitly not a place where you want to put technical details. This should be reserved to some other system, this is a high level overview on what the system is doing. Posts are built to be human readable and human sounding, to avoid boring the readers and to ensure that people actually use this.
Thoughts?
Friday, August 29, 2008
#
Recursive Meta Programming
I am currently writing a DSL that is used to meta program another DSL that is used to to do some action (it is actually turtles 7 layers deep, but we will skip that). It gets to be fairly interesting, although trying to draw that as a diagram is... a bit challenging.
Oh, and there are at least a few parts that rewrite itself.
Ever tried to do incremental method munging? That is when you take code from several places and start applying logic to where to put it. Only useful because of a lot of interesting constraints that we have to deal with in this project, and probably will be actively harmful on other scenarios. And that is only one technique that I am using there.
But a damn elegant approach to solve a problem, wow!
Thinking about chapter 12 - DSL Patterns
I am giving a lot of thought to this chapter, because I want to be able to throw out as much best & worst practices as I can to the reader. Here is what I have right now:
- Auditable DSL - Dealing with the large scale - what the hell is going on?
- User extensible languages
- Multi lingual languages
- Multi file languages
- Data as a first class concept
- Code == Data == Code
- Strategies for editing DSL in production
- Code data mining
- DSL dialects
I am still looking for the tenth piece...
A legacy of conspiracy
Contrary to popular opinion, I have not been kidnapped, nor have I been hit on the head, nor have I started to seek that kind of job security.
I gave a talk about legacy code and refactoring, and I needed something concrete to talk about. Unfortunately, most legacy code is too intertwined to be able to extract out in order to talk about it in isolation. So I set out to write my own.
To all the people who assume that I don't live in the real world, or that I don't deal with legacy systems... well, I think that this code shows that I do know what is going out there.
And just to make it clear, no, I wouldn't write this type of code for any reason. Not for a spike or for a lark. But I think that this is a pretty good archeological fake, even if I say so myself.
DSL Dialects
Let us take this fancy DSL:
And let us say that we want to give the user some sort of UI that shows how this DSL works. The implementation of this DSL isn't really friendly for the UI. It was built for execution, not for display.
So how are we going to solve the problem? There are a couple of ways of doing that, but the easiest solution that I know of consists of creating a new language implementation that is focused on providing an easy to build UI. A dialect can be either a different language (or version of the language) that maps to the same backend engine, or it can be a different engine that is mapped to the same language.
This is part of the reason that it is so important to create strict separation between the two.
Wednesday, August 27, 2008
#
What I am working on...
I am just going to post that, and watch what happens. I will note that this is code that I just wrote, from scratch.
public class TaxCalculator
{
private string conStr;
private DataSet rates;
public TaxCalculator(string conStr)
{
this.conStr = conStr;
using (SqlConnection con = new SqlConnection(conStr))
{
con.Open();
using (SqlCommand cmd = new SqlCommand("SELECT * FROM tblTxRtes", con))
{
rates = new DataSet();
new SqlDataAdapter(cmd).Fill(rates);
Log.Write("Read " + rates.Tables[0].Rows.Count + " rates from database");
if (rates.Tables[0].Rows.Count == 0)
{
MailMessage msg = new MailMessage("important@legacy.org", "joe@legacy.com");
msg.Subject = "NO RATES IN DATABASE!!!!!";
msg.Priority = MailPriority.High;
new SmtpClient("mail.legacy.com", 9089).Send(msg);
Log.Write("No rates for taxes found in " + conStr);
throw new ApplicationException("No rates, Joe forgot to load the rates AGAIN!");
}
}
}
}
public bool Process(XmlDocument transaction)
{
try
{
Hashtable tx2tot = new Hashtable();
foreach (XmlNode o in transaction.FirstChild.ChildNodes)
{
restart:
if (o.Attributes["type"].Value == "2")
{
Log.Write("Type two transaction processing");
decimal total = decimal.Parse(o.Attributes["tot"].Value);
XmlAttribute attribute = transaction.CreateAttribute("tax");
decimal r = -1;
foreach (DataRow dataRow in rates.Tables[0].Rows)
{
if ((string)dataRow[2] == o.SelectSingleNode("//cust-details/state").Value)
{
r = decimal.Parse(dataRow[2].ToString());
}
}
Log.Write("Rate calculated and is: " + r);
o.Attributes.Append(attribute);
if (r == -1)
{
MailMessage msg = new MailMessage("important@legacy.org", "joe@legacy.com");
msg.Subject = "NO RATES FOR " + o.SelectSingleNode("//cust-details/state").Value + " TRANSACTION !!!!ABORTED!!!!";
msg.Priority = MailPriority.High;
new SmtpClient("mail.legacy.com", 9089).Send(msg);
Log.Write("No rate for transaction in tranasction state");
throw new ApplicationException("No rates, Joe forgot to load the rates AGAIN!");
}
tx2tot.Add(o.Attributes["id"], total * r);
attribute.Value = (total * r).ToString();
}
else if (o.Attributes["type"].Value == "1")
{
//2006-05-02 just need to do the calc
decimal total = 0;
foreach (XmlNode i in o.ChildNodes)
{
total += ProductPriceByNode(i);
}
try
{
// 2007-02-19 not so simple, TX has different rule
if (o.SelectSingleNode("//cust-details/state").Value == "TX")
{
total *= (decimal)1.02;
}
}
catch (NullReferenceException)
{
XmlElement element = transaction.CreateElement("state");
element.Value = "NJ";
o.SelectSingleNode("//cust-details").AppendChild(element);
}
XmlAttribute attribute = transaction.CreateAttribute("tax");
decimal r = -1;
foreach (DataRow dataRow in rates.Tables[0].Rows)
{
if ((string)dataRow[2] == o.SelectSingleNode("//cust-details/state").Value)
{
r = decimal.Parse(dataRow[2].ToString());
}
}
if (r == -1)
{
MailMessage msg = new MailMessage("important@legacy.org", "joe@legacy.com");
msg.Subject = "NO RATES FOR " + o.SelectSingleNode("//cust-details/state").Value + " TRANSACTION !!!!ABORTED!!!!";
msg.Priority = MailPriority.High;
new SmtpClient("mail.legacy.com", 9089).Send(msg);
throw new ApplicationException("No rates, Joe forgot to load the rates AGAIN!");
}
attribute.Value = (total * r).ToString();
tx2tot.Add(o.Attributes["id"], total * r);
o.Attributes.Append(attribute);
}
else if (o.Attributes["type"].Value == "@")
{
o.Attributes["type"].Value = "2";
goto restart;
// 2007-04-30 some bastard from northwind made a mistake and they have 3 months release cycle, so we have to
// fix this because they won't until sep-07
}
else
{
throw new Exception("UNKNOWN TX TYPE");
}
}
SqlConnection con2 = new SqlConnection(conStr);
SqlCommand cmd2 = new SqlCommand();
cmd2.Connection = con2;
con2.Open();
foreach (DictionaryEntry d in tx2tot)
{
cmd2.CommandText = "usp_TrackTxNew";
cmd2.Parameters.Add("cid", transaction.SelectSingleNode("//cust-details/@id").Value);
cmd2.Parameters.Add("tx", d.Key);
cmd2.Parameters.Add("tot", d.Value);
cmd2.ExecuteNonQuery();
}
con2.Close();
}
catch (Exception e)
{
if (e.Message == "UNKNOWN TX TYPE")
{
return false;
}
throw e;
}
return true;
}
private decimal ProductPriceByNode(XmlNode item)
{
using (SqlConnection con = new SqlConnection(conStr))
{
con.Open();
using (SqlCommand cmd = new SqlCommand("SELECT * FROM tblProducts WHERE pid=" + item.Attributes["id"], con))
{
DataSet set = new DataSet();
new SqlDataAdapter(cmd).Fill(set);
return (decimal)set.Tables[0].Rows[0][4];
}
}
}
}
Legacy Driven Development
Here is an interesting problem that I run into. I needed to produce an XML document for an external system to consume. This is a fairly complex document format, and there are a lot of scenarios to support. I began to test drive the creation of the XML document, but it turn out that I kept having to make changes as I run into more scenarios that invalidated previous assumptions that I made.
Now, we are talking about a very short iteration cycle, I might write a test to validate an assumption (attempting to put two items in the same container should throws) and an hour later realize that it is a legal, if strange, behavior. The tests became a pain point, I had to keep updating things because the invariant that they were based upon were wrong.
At that point, I decided that TDD was exactly the wrong approach for this scenario. Therefor, I decided that I am going to fall back to the old "trial and error" method. In this case, producing the XML and comparing using a diff tool.
The friction in the process went down significantly, because I didn't have to go and fix the tests all the time. I did break things that used to work, but I caught them mostly with manual diff checks.
So far, not a really interesting story. What is interesting is what happens when I decided that I have done enough work to consider most scenarios to be completed. I took all the scenarios and started generating tests for those. So for each scenario I now have a test that tests the current behavior of the system. This is blind testing. That is, I assume that the system is working correctly, and I want to ensure that it keeps working in this way. I am not sure what each test is doing, but the current behavior is assumed to be correct until proven otherwise..
Now I am back to having my usual safety net, and it is a lot of fun to go from zero tests to nearly five hundred tests in a few minutes.
This doesn't prove that the behavior of the system is correct, but it does ensure no regression and make sure that we have a stable platform to work from. We might find a bug, but then we can fix it in safety.
I don't recommend this approach for general use, but for this case, it has proven to be very useful.
Code Data Mining
I just wrote this piece of code:
class ExpressionInserterVisitor : DepthFirstVisitor
{
public override bool Visit(Node node)
{
using(var con = new SqlConnection("data source=localhost;Initial Catalog=Test;Trusted_Connection=yes"))
using (var command = con.CreateCommand())
{
con.Open();
command.CommandText = "INSERT INTO Expressions (Expression) VALUES(@P1)";
command.Parameters.AddWithValue("@P1", node.ToString());
command.ExecuteNonQuery();
}
Console.WriteLine(node);
return base.Visit(node);
}
}
As you can imagine, this is disposable code, but why did I write that?
I run this code on the entire DSL code base that I have, and then started applying metrics to it. In particular, I was interested in trying to find repeated concepts that has not been codified.
For example, if this would have shown 7 uses of:
user.IsPreferred and order.Total > 500 and (order.PaymentMethod is Cash or not user.IsHighRisk)
Then this is a good indication that I have a business concept waiting to be discovered here, and I turn that into a part of my language:
IsGoodDealForVendor (or something like that)
Here we aren't interested in the usual code quality metrics, we are interested in business quality metrics :-) And the results were, to say the least, impressive.
Didja know? #Develop has a class designer
This is a sample:
I am finding #Develop to be a treasure throve of components that can be used for DSL development... with the added bonus of being accessible and easy to use.
JFHCI: Quote of the Day
This is probably the best quote that I can find about the motivation for JFHCI:
Enabling developers to write "bad" code quickly and safely.