+972 52-548-6969
M is to DSL as Drag & Drop is to programming
I have been sitting on this post for a while now, because that was my first impression after seeing Oslo & M and all the hype around it from the PDC. To be frank, I had a hard time believing my own gut feeling. I kept having the feeling that I am missing something, which is why I avoided talking about this so far.
But, as time passed, and as we started to see more and more about Oslo and M, it validated my initial thinking. Now, just to be clear, I don’t intend to even touch on the whole of Oslo in this post. I don’t have a problem stating that I still don’t see the whole point there, but that is beside the point (no pun intended). What I would like to talk about is the M language, its usage, and the DSL that Microsoft shows as samples.
I see M as a whole lot of effort trying to optimize something that is really not that interesting, complex or really very hard. I look at the M language, the way that you worked with, the tooling and the API and I would fully agree that it is a nice parser generator.
What it is not, I have to say, is a DSL toolkit. It is just one, small, part of building a DSL. And, to be perfectly honest, M is the drag & drop of DSL. It looks good, on first glance, but then you dig just a little deeper and you see what actually going on, and you realize that you are probably not where you wanted to be.I see it as trying very hard to optimize opening the car’s door. While I assume that this is interesting to someone in the world, optimizing the opening of a car door is crucial, I don’t really see it as an important feature. More to the point, it has negligible effect on the time taken for the primary task for which we use a car, the actual driving!
Why am I saying that?
Well, M is used for defining the syntax of the language, which is what most people look at. It does a good job there, but it also stops there. And there is a lot of stuff other then the syntax that you really care about.
Here is a snippet from MisBehave, which was an attempt to build a BDD framework on top of M:
Pretty impressive syntax, right?
The problem is that there isn’t really a good way to take this and translate that into something that is executable. Not without doing a lot of work. And that is why I am saying that M isn’t really an important piece of the stack. The actual syntax definition isn’t really that important. It is all the other things that you do with the DSL that matters.
Let us take a look at MUrl:
I am looking at this, and after looking at the source code, I still can’t figure out the point.
Yes, this is a demo DSL. But it is a good example that shows how you can completely miss the point with regards to a DSL. What problem does this DSL solve? What benefits do I get from integrating that into my system?
How does this helps me solve a real problem?
The answer is that it doesn’t. The only remotely useful case that I can think of is if I really want to be able to issue REST calls from the command line, and even then, there are better ways of doing that on the command line. MUrl is an exercise in abstraction for the sake of abstraction. More than that, it gives the impression that it is something that is is not.
If you want to show me a DSL, show me one that has logic, not one that is a glorified serialization format. That is the sweet spot for a DSL, to extract policy decisions from your systems, so you can work with them at a higher level and have easier time making change.
M is not a language for creating DSL. It is a language to define a serialization format, that is all.
Comments
To a certain extent, I agree with this. M does transform an input and returns an output in the form of structured data no matter which way you cut it. I have been using M (specifically MGrammar) on bdUnit ( usingsystembeginner.blogspot.com/.../...using.html) and I find that I do need to carry out a lot of work in backend C# to deserialize the structured data and then manipulate the types/objects that this produced.
However, MGrammar does give fine grained control over how an input is tokenised and transformed/serialized. I chose MGrammar/M/Oslo because of the control it gave over the input syntax without too many compromises. I believe this is a key requirement when creating an external DSL, especially one which needs to be 'Business-Readable' and 'Business-Writable'.
I have created a project for translating MGraph into executable code easily:
http://metasharp.codeplex.com/
sample:
metasharp.codeplex.com/.../View.aspx
Does M have any advantages over ANTLR's tree transforms for creating and manipulating AST? That stuff was very handy when building query DSL.
(Disclaimer: I know nothing about DSLs)
Can M be used in combination with any other DSL toolkit? Do other DSL toolkits even exist? What I mean here is, can you use M for what it's good for, or are there serious drawbacks?
I totally agree with you Oren. When I first peeked at M I thought it was a serialization format. This was primarily because of the "Oslo" repository thing, and models being stored in the database.
When I looked at the demo code, I realized that you could use M to define grammars also.
I still don't understand why people are amazed by M. M is nothing special, and even if you have some tool support, it doesn't come close that what more mature parser/generators like ANTLR has.
And I really don't get the entire "Oslo" vision, what is that all about?
I haven't done much with M, but I have been doing a fair bit with Mgrammar here lately and I would have to concur. Mgrammar is a nice tool for generating parsers but so is ANTLRv3. Still, at the end of the day you have to do something with the parse tree and that generally means writing some code. Perhaps what's needed is a "meta language", something that has concise syntax for pattern matching and operating 'over tree structures (thinly veiled plug for F#).
In some ways, I think Intellipad is actually the most interesting part of "Oslo" so far.
Ryan,
Not that I easily see.
The syntax seems easier for the simplest stuff, and it has builtin support for VS.
ANTLR has similar support for eclipse.
configurator
Not easily, no
Thomas,
FWIW, ANTLR has tree parsing syntax.
I'm not very familiar with ANTLR but I think the difference is just that MGrammar parses a tree and gives you objects at runtime (MGraph) while ANTLR (I'm not 100% on this) generates code that needs to be compiled.
There is a big difference between M and MGrammar too...
I would hope that the story of Oslo has not been told to completion. M seems to be the bit that is most presentable. I don't think that people are that amazed at Oslo, but I have hopes that the working with models will be made easier with an offering like Oslo.
Ayende Rahien: I think you are missing a number of key points about "M" that I would be happy to discuss with you. If you have seen the video from our MIX talk ( http://videos.visitmix.com/MIX09/T11F), we covered some of that at a cursory level, but there is a lot of detail that you may be interested in. I am happy to have an email thread or a call to discuss. Shoot me email at douglasp@microsoft.com if you are interested.
BTW: You may want to read blog.jclark.com/.../getting-involved-with-m.html (he was very involved with the formation of the entire XML stack), for his view.
I got the feeling writing the DSL in M would give you a text editor for the language (inellisense, etc), "for free". I wonder how does M stack vs MPS( http://www.jetbrains.com/mps/index.html) ? So many interesting things, so little time ...
I can sort of see the usefulness in MGrammar for building external DSLs, how it compares to ANTLR or GoldParser I am not sure.
But doing something useful with MGrammar requires a lot of work (on the runtime), maybe Justins MetaSharp could help there.
Anyway I share your confusion of where Microsoft is going with this, and what the real problems are that they are trying to solve. Especially confusing is the whole M modeling and SQL schema generation, M Repository, M model data driven by behavior defined in workflows etc.
It would be great with a full source example of a real application to show exactly how they envision this being used for real applications.
Wile looking for something else, I stumbled on a quote attributed to Don Box yesterday in the Oslo forums that does confirm to some extent that the story isn't fully told here. I'll try to dig up the link and post it back here in a couple of hours.
The short of it was there was still work going on in unifying Mgrammar and M so that there is a direct mapping from "things" specified in the DSL to the model. This would cut down to some extent the code I'm finding myself writing, particularly as the tooling ecosystem that will develop around M appears (putting some things in the DB just to query them back and execute it later seems a little alien),
As any show biz star you have to rely on noise to get the publicity that supports your business.
"If you want to show me a DSL, show me one that has logic, not one that is a glorified serialization format. That is the sweet spot for a DSL, to extract policy decisions from your systems, so you can work with them at a higher level and have easier time making change."
Couldn't agree with you more, the whole XSD discussion at PDC was ridiculous and they do seem to have gone overboard on the model is data and all behavior is (Windows) workflow idea. There may be scope for it to be a great idea, especially when used over an existing model, but so far the story has left me extremely cold.
Anyway I'd still recommend this blog: http://tinyfinger.blogspot.com/
At the PDC and since I've spoken with Don, Chris Sells, and Glenn Block about the possibility of them helping Greg Leake in doing an Oslo/MEF/Unity port of his .NET StockTrader reference app which I suggested might make a good candidate for an Oslo reference app of some substance. All of them sounded pretty up for doing that and providing Greg with whatever resources he needed to make that happen. When I finally got ahold of Greg, which took awhile, he said he definitely wanted to take it to 4.0 and Oslo as well - so hopefully it will as I agree a reference app of some substance really needs to be put out there as a vehicle to get the essential ideas across.
MisBehave has taken its syntax from the Ruby BDD tool Cucumber. To see what problem it solves, go to http://cukes.info/ .
Yes, like Antti mentioned above, MisBehave takes the path opened by rspec/cucumber, which is the approach suggested by Dan North to define a language for defining executable specifications.
The story snippet that you mentioned is taken out of context. It is a descriptive snippet and should be matched with the relevant scenarios that DO eventually map to executable code. We regularly write automated acceptance tests this way (using cucumber), so it addresses a real world problem. I haven't "finished" Misbehave simply because I wanted to play with MGrammar rather than provide a fully working framework. If enough people scream about it, I'll build a fully working version to match cucumber. The question however should be: does MGrammar add anything that GoldParser does not have? I've never used GoldParser before, but I since learned that they are EXTREMELY similar, at least so far, so maybe it is not a big deal after all. Perhaps MGrammar is a small piece of the Oslo vision though - only time will tell.
The 'whole picture' is scattered amongst a number of emerging technologies and paradigms. Oslo appears to be attempting to grasp all of these, or at least the lion's share of them, into a single thing called Microsoft Oslo.
THE core concept is: increase productivity
via some derivative concepts:
pre-built frameworks - a pre-package architecture into which you apply, ...
guided customization - this is a lengthy topic, basically you "limit" customization, or "guide" it via provided conventions and constraints
optimized development methodology - I hate to use the term "model-driven", b/c pre-used terms conjure up old ideas.
A practical example would be Ruby on Rails.
However, there are larger app spaces to consider, e.g.
MetaCase - a graphical modeling tool, and quite mature in terms of paradigm, practice and results.
openArchitectureWare - too 'techie', Microsoft will lower the skills bar vs. this as it is today. (However, one could build on top of XText/et.al. in the same way one would M or MGrammar. For example, non-techncal users can be enabled to build artifacts (e.g. specs) that contribute to the SDLC directly. See http://cukes.info/ for another example.
When it comes to M, it falls under this 'domain-specific language' umbrella. The problem with the 'DSL' topic is it's "scattered", too. DSLs come in so many forms, you might as well say they're anything BUT general-purpose programming languages (excluding those that intrinsically enable/support them, "internal DSLs" as Martin Fowler (I believe) coined it).
The 'big picture' for Oslo is a very wide mosaic. That's why most aren't getting it right away.
This is only one small way to use a DSL. And, it's not a good example, I'm afraid, of the potential of Oslo. Perhaps Microsoft is slow to real-world examples, because they're tackling a space that is much wider than anything else in a given area Oslo is involved with (e.g. model-driven development, pre-built frameworks, guidance and more). When it's done, it'll probably be a distilled unification of a number of practices and paradigms going on in the market today, but under one umbrella.
DSLs are executable in nature and declarative in nature. You don't have to have one that 'has logic' to be a DSL. Many DSLs are no more than configuration information for a pre-built framework. Commercial-grade scenarios typically work with a suite of DSLs in a single space, e.g. one that provides the web framework configuration/customization, one that provides the app security, another that provides the ORM mapping, another for biz rules and the list goes on. All of these can be in a single web app, for example, working together to rapidly build a complex commercial-grade solution by parsing DSL info into a repository, which is extracted by everything from code generators to middleware to pre-built frameworks as cfg data.
Comment preview