One of the things that you keep hearing about Microsoft products is how much time and effort is dedicated to ensuring Backward Compatibility. I have had a lot of arguments with Microsoft people about certain design decisions that they have made, and usually the argument ended up with “We have to do it this was to ensure Backward Compatibility”.

That sucked, but at least I could sleep soundly, knowing that if I had software running on version X of Microsoft, I could at least be pretty sure that it would work in X+1.

Until Entity Framework 4.1 Update 1 shipped, that is. Frans Bouma has done a great job in describing what the problem actually is, including going all the way and actually submitting a patch with the code required to fix this issue.

But basically, starting with Entity Framework 4.1 Update 1 (the issue also appears in Entity Framework 4.2 CTP, but I don’t care about this much now), you can’t use generic providers with Entity Framework. Just to explain, generic providers is pretty much the one way that you can integrate with Entity Framework if you want to write a profiler or a cacher or a… you get the point.

Looking at the code, it is pretty obvious that this is not intentional, but just someone deciding to implement a method without considering the full ramifications. When I found out about the bug, I tried figuring out a way to resolve it, but the only work around would require me to provide a different assembly for each provider that I wanted to support (and there are dozens that we support on EF 4.0 and EF 3.5 right now).

We have currently implemented a work around for SQL Server only, but if you want to use Entity Framework Profiler with Entity Framework 4.1 Update 1 and a database other than SQL Server, we would have to create a special build for your scenario, rather than have you use the generic provider method, which have worked so far.

Remember the beginning of this post? How I said that Backward Compatibility is something that Microsoft is taking so seriously?

Naturally I felt that this (a bug that impacts anyone who extends Entity Framework in such a drastic way) is an important issue to raise with Microsoft. So I contacted the team with my finding, and the response that I got was basically: Use the old version if you want this supported.

What I didn’t get was:

• Addressing the issue of a breaking change of this magnitude that isn’t even on a major release, it is an update to a minor release.
• Whatever they are even going to fix it, and when this is going to be out.
• Whatever the next version (4.2 CTP also have this bug) is going to carry on this issue or not.

I find this unacceptable. The fact that there is a problem with a CTP is annoying, but not that important. The fact that a fully release package has this backward compatibility issue is horrible.

What makes it even worse is the fact that this is an update to a minor version, people excepts this to be a safe upgrade, not something that requires full testing. And anyone who is going to be running Update-Package in VS is going to hit this problem, and Update-Package is something that people do very often. And suddenly, Entity Framework Profiler can no longer work.

Considering the costs that the entire .NET community has to bear in order for Microsoft to preserve backward compatibility, I am deeply disappointed that when I actually need this backward compatibility.

This is from the same guys that refused (for five years!) to fix this bug:

new System.ComponentModel.Int32Converter().IsValid("yellow") == true

Because, and I quote:

We do not have the luxury of making that risky assumption that people will not be affected by the potential change. I know this can be frustrating for you as it is for us. Thanks for your understanding in this matter.

Let me put this to you in perspective, anyone who is using EF Prof is likely to (almost by chance) to get hit by that. When this happens, our only option is to tell them to switch back a version?

That makes us look very bad, regardless of what is the actual reason for that. That means that I am having to undermine my users' trust in my product. "He isn't supporting 4.1, and he might not support 4.2, so we probably can't buy his product, because we want to have the newest version".

This is very upsetting. Not so much about the actual bug, those happen, and I can easily imagine the guy writing the code making assumptions that turn out to be false. Heavens know that I have done the same many times before. I don’t even worry too much about this having escaped the famous Microsoft Testing Cycle.

What (to be frank) pisses me off is that the response that we got from Microsoft for this was that they aren’t going to fix this. That the only choice that I have it to tell people to downgrade if they want to use my product (with all the implications that has for my product).

I was forwarded this link http://microsoftnlayerapp.codeplex.com/ with a request to review it. I thought about skipping it until I saw:

The main problem here?

If it is a simple business scenario, it doesn’t need DDD.

Now, just to be clear on this sample app. It is not someone working on their free time, this is something that is Official Guidance (shows up at: http://msdn.microsoft.com/es-es/architecture/en) and people are expected to read and follow this.

I haven’t reviewed the code yet, but I am scared just by looking at the architecture diagram.

This is going to be my last LightSwitch post for a while.

I wanted to talk about something that I found which was at once both very surprising and Doh! at the same time.

Take a look here:

What you don’t know is this was generated from a request similar to this one:

wget http://localhost:22940/Services/LSTest-Implementation-ApplicationDataDomainService.svc/binary/AnimalsSet_All?$orderby=it.Id&$take=45&$includeTotalCount=

What made me choke was that the size of the response for this was 2.3 MB.

Can you guess why?

The image took up most of the data, obviously. In fact, I just dropped an image from my camera, so it was a pretty big one.

And that lead to another problem. It is obviously a really bad idea to send that image on the wire all the time, but LightSwitch make is so easy, indeed, even after I noticed the size of the request, it took me a while to understand what exactly is causing the issue.

And there doesn’t seems to be any easy way to tell LightSwitch that we want to put the property here, but only load it in certain circumstances. For that matter, I would generally want to make the image accessible via HTTP, which means that I gain advantages such as parallel downloads, caching, etc.

But there doesn’t seems to be any (obvious) way to do something as simple as binding a property to an Image control’s Url property.

Something that I found many high level tools are really bad at is source control, so I thought that I would give LightSwitch a chance there.

I created a Git repository and shoved everything into it, then I decided that I would rename a property and see what is going on.

I changed the Animals.Species to Animals.AnimalType, which gives me:

This is precisely what I wanted to see.

Let us see what happen when I add a new table. And that created a new set in the ApplicationDefinition.lsml file.

Overall, this is much better than I feared.

I am still concerned about having everything in a single file (which is a receipt for having a lot of merge conflicts), but at least you can diff & work with it, assuming that you know how the file format works, and is seems like it is at least a semi reasonable one.

Nevertheless, as promised:

True story, I used to have a lot of ravens in my backyard, but they seem to have gone away single my dog killed one of them, about a week after RavenDB’s launch.

I thought it would be a good idea to see what sort of data access behavior LightSwitch applications have. So I hook it up with the EntityFramework Profiler and took it for a spin.

It is interesting to note that it seems that every operation that is running is running in the context of a distributed transaction:

There is a time & place to use DTC, but in general, you should avoid them until you really need them. I assume that this is something that is actually being triggered by WCF behavior, not intentional.

Now, let us look at what a simple search looks like:

This search results in:

That sound? Yes, the one that you just heard. That is the sound of a DBA somewhere expiring. The presentation about LightSwitch touted how you can search every field. And you certainly can. You can also swim across the English channel, but I found that taking the train seems to be an easier way to go about doing this.

Doing this sort of searching is going to be:

• Very expensive once you have any reasonable amount of data.
• Prevent usage of indexes to optimize performance.

In other words, this is an extremely brute force approach for this, and it is going to be pretty bad from performance perspective.

Interestingly, it seems that LS is using optimistic concurrency by default.

I wonder why they use the slowest method possible for this, instead of using version numbers.

Now, let see how it handles references. I think that I run into something which is a problem, consider:

Which generates:

This make sense only if you can think of the underlying data model. It certainly seems backward to me.

I fixed that, and created four animals, each as the parent of the other:

Which is nice, except that here is the SQL required to generate this screen:

-- statement #1
SELECT [GroupBy1].[A1] AS [C1]
FROM   (SELECT COUNT(1) AS [A1]
        FROM   [dbo].[AnimalsSet] AS [Extent1]) AS [GroupBy1]

-- statement #2
SELECT   TOP ( 45 ) [Extent1].[Id]              AS [Id],
                    [Extent1].[Name]            AS [Name],
                    [Extent1].[DateOfBirth]     AS [DateOfBirth],
                    [Extent1].[Species]         AS [Species],
                    [Extent1].[Color]           AS [Color],
                    [Extent1].[Pic]             AS [Pic],
                    [Extent1].[Animals_Animals] AS [Animals_Animals]
FROM     (SELECT [Extent1].[Id]                      AS [Id],
                 [Extent1].[Name]                    AS [Name],
                 [Extent1].[DateOfBirth]             AS [DateOfBirth],
                 [Extent1].[Species]                 AS [Species],
                 [Extent1].[Color]                   AS [Color],
                 [Extent1].[Pic]                     AS [Pic],
                 [Extent1].[Animals_Animals]         AS [Animals_Animals],
                 row_number()
                   OVER(ORDER BY [Extent1].[Id] ASC) AS [row_number]
          FROM   [dbo].[AnimalsSet] AS [Extent1]) AS [Extent1]
WHERE    [Extent1].[row_number] > 0
ORDER BY [Extent1].[Id] ASC

-- statement #3
SELECT [Extent1].[Id]              AS [Id],
       [Extent1].[Name]            AS [Name],
       [Extent1].[DateOfBirth]     AS [DateOfBirth],
       [Extent1].[Species]         AS [Species],
       [Extent1].[Color]           AS [Color],
       [Extent1].[Pic]             AS [Pic],
       [Extent1].[Animals_Animals] AS [Animals_Animals]
FROM   [dbo].[AnimalsSet] AS [Extent1]
WHERE  1 = [Extent1].[Id]

-- statement #4
SELECT [Extent1].[Id]              AS [Id],
       [Extent1].[Name]            AS [Name],
       [Extent1].[DateOfBirth]     AS [DateOfBirth],
       [Extent1].[Species]         AS [Species],
       [Extent1].[Color]           AS [Color],
       [Extent1].[Pic]             AS [Pic],
       [Extent1].[Animals_Animals] AS [Animals_Animals]
FROM   [dbo].[AnimalsSet] AS [Extent1]
WHERE  2 = [Extent1].[Id]

-- statement #5
SELECT [Extent1].[Id]              AS [Id],
       [Extent1].[Name]            AS [Name],
       [Extent1].[DateOfBirth]     AS [DateOfBirth],
       [Extent1].[Species]         AS [Species],
       [Extent1].[Color]           AS [Color],
       [Extent1].[Pic]             AS [Pic],
       [Extent1].[Animals_Animals] AS [Animals_Animals]
FROM   [dbo].[AnimalsSet] AS [Extent1]
WHERE  3 = [Extent1].[Id]

I told you that there is a select n+1 builtin into the product, now didn’t I?

Now, to make things just that much worse, it isn’t actually a Select N+1 that you’ll easily recognize. because this doesn’t happen on a single request. Instead, we have a multi tier Select N+1.

What is actually happening is that in this case, we make the first request to get the data, then we make an additional web request per returned result to get the data about the parent.

And I think that you’ll have to admit that a Parent->>Children association isn’t something that is out of the ordinary. In typical system, where you may have many associations, this “feature” alone is going to slow the system to a crawl.

I found this post interesting, so I thought that I would try to answer it.

If I run Dev Div, I would:

• Utilize CodePlex Foundation to a much higher degree. Right now CPF manages about 6 projects, all of them seems to originate from Microsoft. None of them are actually useful for CPF in the sense of creating an excitement around it. If I run Dev Div, and given the indication of the lack of will from within Microsoft to maintain the DLR, I would give the CPF the DLR, along with a small budget (1 million annually should more than cover that) and set it free. That 1 million? I would put it down as marketing, because the amount of good will that you’ll generate from doing that is enormous. And the level of publicity that you’ll get for CPF is huge.

• Sit down with marketing and create a new set of personas, based on what is actually happening in the real world. Mort, Einstein and that Elvis dude might be good when you are building the product and designing the API, but there are other aspects to consider, and acceptance in the community is important. A lot of the public blunders that Microsoft has made recently has been just that, marketing mistakes more than anything else.

• Build for each project a set of scenarios that appeal to each group. For the PHP crowd, point out how WebMatrix makes their life simple. For the ALT.Net crowd, show them how they can use IIS Express to run their unit tests without requiring anything installed. Right now what is going on is that pushing to one side alienate the other.
  Let me put this another way, when someone asks me why use NHibernate instead of Entity Framework, I am going to inform them that Microsoft continues their trend of new Data Access framework every two years, and they have gone retro with Microsoft.Data.

• Kill duplicated effort before it goes out the door. Linq to SQL & Entity Framework are a great example of things that should have been developed concurrently (because either project had a chance of failure, so hedging your bets is good, if you can afford that), but they should have never been allowed to both see the light of day. The marketing damage from moving Linq to SQL to maintenance mode was huge.

• Simplify, simplify, simplify! Microsoft does a lot of good things, but they tend to provide solutions that are wide scoped and complex. Often enough, so complex that people just give up on them. Case in point, if at all possible, I wouldn’t use WCF for communication. I would rather role my own RPC stack, because that makes things easier. That is a Fail for Microsoft in many cases, and one of the reasons that they are leaking developers to other platforms.

• Forbid developers from arguing with customers publically about policy decisions. The reasoning is simple, devs don’t have a lot (if any) impact on policy decisions, they can explain them, sure. But when there is disagreement, having an argument about it tends to end up with bad blood in the water. Such discussions should be done at a level that can actually do something. Case in point, recent WebMatrix arguments in twitter & blogs.

• Focus some love & attention toward the non entry level segment of the market. There has been a worrying set of events recently that indicate a shift in focus toward the entry level market, and the rest of developers are feeling abandoned. That shouldn’t be allowed to remain this way for long.

If you’ll not, except for the suggestion about turning simplification to an art form and applying it ruthlessly, most of the suggestions are actually about marketing and positioning, not about product development.

Microsoft LightSwitch is a new 4GL tool from Microsoft, this is another in the series of “you don’t have to write any code” tools that I have seen.

Those are the tools that will give the secretary the ability to create applications and eliminate the need for coders. The industry has been chasing those tools since the 80s (does anyone remember the promises of the CASE tools?). We have seen many attempts at doing this, and all of them have run into a wall pretty quickly.

Oh, you can build a tool that gives you UI on top of a data store pretty easily. And you can go pretty far with it, but eventually your ability to point & click hit the limit, and you have to write code. And that is things totally breaks down.

LightSwitch is not yet publically available, so I have to rely on the presentation that Microsoft published. And I can tell you that I am filled with dread, based on what I have seen.

First of all, I strongly object to the following slide. Because I have the experience to know that working with a tool like that is akin to do back flips with a straightjacket on.

The capabilities of the tools that were shown in the presentation have strongly underwhelmed me in terms of newness, complexity or applicability.

Yeah, a meta data driven UI. Yeah, it can do validation on a phone number automatically (really, what happen with my Israeli based phone number?), etc. What is worse, even through the demo, I get the very strong feeling that the whole things is incredibly slow, you can see in the presentation multi second delays between screen repaints.

Then there are things like “it just works as a web app or windows app” which is another pipe dream that the industry has been chasing for a while. And the only piece of code that I have seen is this guy:

Which makes me want to break down and cry.

Do you know why? Because this is going to be the essence of a SELECT N+1 in any system, because this code is going to run once per each row in the grid. And when I can find bugs from watching a presentation, you know that there are going to be more issues.

So, just for fun sake, since I don’t have the bits and I can rely only on the presentation, I decided to make a list of all the things that are likely to be wrong with LightSwitch.

I’ll review it when it comes out, and if it does manage to do everything that it does and still be a tool usable by developers, I’ll have to eat crow (well, Raven :-) ), but I am not overly worried.

Here are a few areas where I am feeling certain things will not work right:

• Source control – how do I diff two versions of the app to see what changes? Are all changes diffable?
• Programmatic modifications:
• what happen when I want to write some code to do custom validation of property (for instance, calling a web service)?
• what happen when I want to put a custom control on the screen (for instance, a google maps widget)?
• Upsizing – when it gets to a 1,000 users and we need a full blown app, how hard it is to do?
• Performance – as I said, I think it is slow from the demo.
• Data access behavior – from what I have seen so far, I am willing to be that it hits its data store pretty heavily.

I fully recognize that there is a need for such a tool, make no mistake. And giving users the ability to do that is important. What I strongly object to is the notion that it would be useful for developers writing real apps, not forms over data. To put it simply, simple forms over data is a solved problem. There is a large number of tools out there to do that. From Access to Oracle Apex to FoxPro. Hell, most CRM solutions will give you just that.

My concern is that there seems to be an emphasis on that being useful for developers as well, and I strongly doubt that.

I just read a very insightful post from Evan Nagle about the topic. In it, Evan hit a key point:

Technically (and commercially) speaking, Microsoft has a slew of perfectly legitimate reasons for splattering the landscape with these "watered down" technologies. But, culturally speaking, it's painful to the professional Microsoft developer. It is. Because the Microsoft developer's personal and professional identity is tied up in the Microsoft stack, and that stack is now catering to a bunch of cat ladies and acne-laden teenagers. That's the real issue here.

If Microsoft thinks that they can get the market for bad developers, good for them. But that needs to be highly segmented. It should be clear that going with that route is leading you to a walled garden and that writing real apps this way is not good. The best comparison I can think of is Access apps in the 90s. There was a clear separation between “just wanna write some forms over data app over the weekend” and “I want to build a real application”. When you built an app in Access, you had very clear idea about the limitations of the application, and you knew that if you wanted something more, it would be a complete re-write.

That was a good thing, because it meant that you could do the quick & dirty things, but when things got messy, you knew that you had to make the jump.

The problem with things like Microsoft.Data is that there is no such line in the sand. And when you call it “Microsoft.*” you give it a seal of approval for everything. And when you have a piece of code that is explicitly designed to support bad coding practices, it is like peeing in the pool. If there is only one pool, it is going to affect everyone. There wouldn’t be nearly as much objection if it was called WebMatrix.Data, because that would clearly put it in someone else’s pool, and it that turn into a putrid swamp, I don’t really care.

There is another issue here, and it is not just that the community is afraid of inheriting horrible Microsoft.Data projects. It is much more than that.

Salary data is from the UK, because that is the first site I found with the numbers)

• C# average salary: £42,000
• VB.NET average salary: £34,000

Now, I really can’t think of a good technical reason why VB.Net programmers are paid so much less, but those data points match what I have seen about salaries for developers in both positions.

In other words, VB.Net developers are getting paid a lot less for doing the same job.

Now, why is that? Is it because of the legacy of VisualBasic still haunts the VB guys? Because there is still the impression that VB is a toy language? I don’t know, but I think that at least in part, that is true. And that is what happen when a platform gets a reputation for being messy. People know in their bones that building stuff there is going to be costly, because maintaining this stuff is so hard.

Microsoft has repeatedly stated that they are after the people who currently go to PHP. Let me do the same comparison:

• PHP average salary: £32,500
• Ruby average salary: £45,000
• Python average salary: £47,500

I don’t like those numbers. I really don’t like them.

Put simply, if Microsoft attempts to make the .NET platform more approachable for the PHP guys, it is also going to devalue the entire platform. I am pretty sure that this is something that they don’t want. Having a lot of hobbist programmer but fewer professional ones is going to hurt the Microsoft eco system.

Microsoft, if you want to create a PHP haven in the .NET world, that is great, but make sure that it is completely isolated, because otherwise you are going to hurt everyone who has a commercial stake in your platform.

I think that there is a lot of sense from commercial point of view in WebMatrix, but it should be clear that this isn’t .NET programming, this is WebMatrix programming. So if Microsoft succeed in gaining market share for this, it won’t be the .NET developers who would suddenly look at a 30% less money, it would be WebMatrix developers who would have to carry that stigma.

This post is copied (with permission) from Roy Osherove. I don’t often do things like that but Roy’s post has pushed a lot of red buttons.

Let me just hand you over to Roy:

A month or so ago, Microsoft Israel started sending out emails to its partners and registered event users to “Save the date!” – Micraoft Teched Israel is coming, and it’s going to be this november!

“Great news” I thought to myself. I’d been to a couple of the MS teched events, as a speaker and as an attendee, and it was lovely and professionally done. Israel is an amazing place for technology and development and TechEd hosted some big names in the world of MS software.

A couple of weeks ago, I was shocked to hear from a couple of people that Microsoft Israel plans to only accept non-MS teched speakers, only from sponsors of the event. That means that according to the amount that you have paid, you get to insert one or more of your own selected speakers as part of teched.

I’ve spent the past couple of weeks trying to gather more evidence of this, and have gotten some input from within MS about this information. It looks like that is indeed the case, though no MS rep. was prepared to answer any email I had publicly. If they approach me now I’d be happy to print their response.

What does this mean?

If this is true, it means that Microsoft Israel is making a grave mistake –

• They are diluting the quality of the speakers for pure money factors. That means, that as a teched attendee, who paid good money, you might be sitting down to watch nothing more that a bunch of infomercials, or sub-standard speakers – since speakers are no longer selected on quality or interest in their topic.
• They are turning the conference from a learning event to a commercial driven event
• They are closing off the stage to the community of speakers who may not be associated with any organization  willing to be a sponsor
• They are losing speakers (such as myself) who will not want to be part of such an event. (yes – even if my company ends up sponsoring the event, I will not take part in it, Sorry Eli!)
• They are saying “F&$K you” to the community of MVPs who should be the people to be approached first about technical talks (my guess is many MVPs wouldn’t want to talk at an event driven that way anyway )

I do hope this ends up not being true, but it looks like it is. MS Israel had already done such a thing with the Developer Days event previouly held in Israel – only sponsors were allowed to insert speakers into the event.

If this turns out to be true I would urge the MS community in Israel to NOT TAKE PART AT THIS EVENT in any form (attendee, speaker, sponsor or otherwise). by taking part, you will be telling MS Israel it’s OK to piss all over the community that they are quietly suffocating anyway.

The MVP case

MS Israel has managed to screw the MVP program as well. MS MVPs (I’m one) have had a tough time here in Israel the past couple of years. ever since yosi taguri left the blue badge ranks, there was not real community leader left. Whoever runs things right now has their eyes and minds set elsewhere, with the software MVP community far from mind and heart. No special MVP events (except a couple of small ones this year). No real MVP leadership happens here, with the MVP MEA lead (Ruari) being on a remote line, is not really what’s needed.

“MVP? What’s that?” I’m sure many MS Israel employees would say. Exactly my point.

Last word

I’ve been disappointed by the MS machine for a while now, but their slowness to realize what real community means in the past couple of years really turns me off. Maybe it’s time to move on. Maybe I shouldn’t be chasing people at MS Israel begging for a room to host the Agile Israel user group. Maybe it’s time to say a big bye bye and start looking at a life a bit more disconnected from that giant. I hear the people at Google are pretty Agile!

And now back to me. I had more discussions in the last two years with Microsoft UK than with Microsoft Israel. I think it says it all, and I am an Israeli MVP who spends most of his time in Israel.

There are some things that I just don’t get. The new MS Courier is one such thing. Check the link, this is basically a book size iPhone/tablet.

Go checkout the video, you’ll notice something that will kill this device.

It uses a pen, to write!

Leaving aside the fact that no OCR program has yet been able to figure out what I am writing (including the one in my brain), using a pen it annoying.

I write about three to four times as fast using a keyboard than using a pen (and I can use both hands). And I don’t write, using archaic pen & paper, much at all. That affect my writing readability when I am using a pen, of course, leading to a feedback cycle.

This pretty much turn me off of the device completely.