Tuesday, January 16, 2007

ECMA approves Open XML as a standard

News item: http://www.betanews.com/article/1165528807

ECMA's press release: http://www.ecma-international.org/news/PressReleases/PR_TC45_Dec2006.htm

An Excerpt:

At the General Assembly meeting on 7 December 2006, Ecma International approved Office Open XML as an Ecma standard (Ecma 376). The General Assembly also approved submitting the standard for adoption under the ISO/IEC JTC 1 process.

The work to standardize OpenXML has been carried out by Ecma International as part of an open, cross-industry collaboration via Technical Committee 45 (Ecma TC45), which includes representatives from Apple, Barclays Capital, BP, The British Library, Essilor, Intel, Microsoft, NextPage, Novell, Statoil, Toshiba, and the United States Library of Congress.

With more and more organizations around the world interested in achieving document processing interoperability, and creating digital archives using open formats, the Office Open XML (Open XML) formats provide an international open standard for word-processing documents, presentations, and spreadsheets that can be freely implemented across multiple applications and platforms, both today and in the future. Multiple vendors such as Corel, Microsoft and Novell already announced implementations of the Open XML standard in very popular applications such as WordPerfect, Open Office and Microsoft Office 2007.

Why programmers shouldn't fear offshoring

Recently, while engaging in my other passion (international relations), I was reading the latest issue of Foreign Affairs, and ran across an interesting essay regarding the increasing outsourcing--or, the term they introduce which I prefer in this case, "offshoring"--of technical work, and I found some interesting analysis there that I think solidifies why I think programmers shouldn't fear offshoring, but instead embrace it and ride the wave to a better life for both us and consumers. Permit me to explain.

The essay, entitled "Offshoring: The Next Industrial Revolution?" (by Alan S. Blinder), opens with an interesting point, made subtly, that offshoring (or "offshore outsourcing"), is really a natural economic consequence:

In February 2004, when N. Gregory Mankiw, a Harvard professor then serving as chairman of the White House Council of Economic Advisers, caused a national uproar with a "textbook" statement about trade, economists rushed to his defense. Mankiw was commenting on the phenomenon that has been clumsily dubbed "offshoring" (or "offshore outsourcing")--the migration of jobs, but not the people who perform them, from rich countries to poor ones. Offshoring, Mankiw said, is only "the latest manifestation of the gains from trade that economists have talked about at least since Adam Smith. ... More things are tradable than were tradable in the past, and that's a good thing." Although Democratic and Republican politicians alike excoriated Mankiw for his callous attitude toward American jobs, economists lined up to support his claim that offshoring is simply international business as usual.

Their economics were basically sound: the well-known principle of comparative advantage implies that trade in new kinds of products will bring overall improvements in productivity and well-being. But Mankiw and his defenders underestimated both the importance of offshoring and its disruptive effect on wealthy countries. Sometimes a quantitative change is so large that it brings qualitative changes, as offshoring likely will. We have so far barely seen the tip of the offshoring iceberg, the eventual dimensions of which may be staggering.

So far, you're not likely convinced that this is a good thing, and Blinder's article doesn't really offer much reassurance as you go on:

To be sure, the furor over Mankiw's remark was grotesquely out of proportion to the current importance of offshoring, which is still largely a prospective phenomenon. Although there are no reliable national data, fragmentary studies indicate that well under a million service-sector jobs have been lost to offshoring to date. (A million seems impressive, but in the gigantic and rapidly churning U.S. labor market, a million jobs is less than two weeks' worth of normal gross job losses.)1 However, constant improvements in technology and global communications will bring much more offshoring of "impersonal services"--that is, services that can be delivered electronically over long distances, with little or no degradation in quality.

That said, we should not view the coming wave of offshoring as an impending catastrophe. Nor should we try to stop it. The normal gains from trade mean that the world as a whole cannot lose from increases in productivity, and the United States and other industrial countries have not only weathered but also benefited from comparable changes in the past. But in order to do so again, the governments and societies of the developed world must face up to the massive, complex, and multifaceted challenges that offshoring will bring. National data systems, trade policies, educational systems, social welfare programs, and politics must all adapt to new realities. Unfortunately, none of this is happening now.

Phrases like "the world cannot lose from increases in productivity" are hardly comforting to programmers who are concerned about their jobs, and hearing "nor should we try to stop" the impending wave of offshoring is not what most programmers want to hear. But there's an interesting analytical point that I think Blinder misses about the software industry, and in order to make the point I have to walk through his argument a bit to get to it. I'm not going to quote the entirety of the article to you, don't worry, but I do have to walk through a few points to get there. Bear with me, it's worth the ride, I think.

Why Offshoring

Blinder first describes the basics of "comparative advantage" and why it's important in this context:

Countries trade with one another for the same reasons that individuals, businesses and regions do: to exploit their comparative advantages. Some advantages are "natural": Texas and Saudi Arabia sit atop massive deposits of oil that are entirely lacking in New York and Japan, and nature has conspired to make Hawaii a more attractive tourist destination than Greenland. Ther eis not much anyone can do about such natural advantages.

But in modern economics, nature's whimsy is far less important than it was in the past. Today, much comparative advantage derives from human effort rather than natural conditions. The concentration of computer companies around Silicon Valley, for example, has nothing to do with bountiful natural deposits of silicon; it has to do with Xerox's fabled Palo Alto Research Center, the proximity of Stanford University, and the arrival of two young men named Hewlett and Packard. Silicon Valley could have sprouted up anywhere.

One important aspect of this modern reality is that patterns of man-made comparative advantage can and do change over time. The economist Jagdish Bhagwait has labeled this phenomenon "kaleidoscopic comparative advantage", and it is critical to understanding offshoring. Once upon a time, the United Kingdom had a comparative advantage in textile manufacturing. Then that advantage shifted to New England, and so jobs were moved from the United Kingdom to the United States.2 Then the comparative advantage in textile manufacturing shifted once again--this time to the Carolinas--and jobs migrated south within the United States.3 Now the comparative advantage in textile manufacturing resides in China and other low-wage countries, and what many are wont to call "American jobs" have been moved there as a result.

Of course, not everything can be traded across long distances. At any point in time, the available technology--especially in transportation and communications4--largely determines what can be traded internationally and what cannot. Economic theorists accordingly divide the world's goods and services into two bins: tradable and non-tradable. Traditionally, any item that could be put in a box and shipped (roughly, manufactured goods) was considered tradable, and anything that could not be put into a box (such as services) or was too heavy to ship (such as houses) was thought of as nontradable. But because technology is always improving and transportation is becoming cheaper and easier, the boundary between what is tradable and what is not is constantly shifting. And unlike comparative advantage, this change is not kaleidoscopic; it moves in only one direction, with more and more items becoming tradable.

The old assumption that if you cannot put it in a box, you cannot trade it is thus hopelessly obsolete. Because packets of digitized information play the role that boxes used to play, many more services are now tradable and many more will surely become so. In the future, and to a great extent already, the key distinction will no longer be between things that can be put in a box and things that cannot. Rather, it will be between services that can be delivered electronically and those that cannot.

Blinder goes on to describe the three industrial revolutions, the first being the one we all learned in school, coming at the end of the 18th century and marked by Adam Smith's The Wealth of Nations in 1776. It was a massive shift in the economic system, as workers in industrializing countries migrated from farm to factory. "It has been estimated that in 1810, 84 percent of the U.S. work force was engaged in agriculture, compared to a paltry 3 percent in manufacturing. By 1960, manufacturing's share had rised to almost 25 percent and agriculture's had dwindled to just 8 percent. (Today, agriculture's share is under 2 percent.)" (This statistic is important, by the way--keep it in mind as we go.) He goes on to point out the second Revolution, the shift from manufacturing to services:

Then came the second Industrial Revolution, and jobs shifted once again--this time away from manufacturing and toward services. The shift to services is still viewed with alarm in the United States and other rich countries, where people bemoan rather than welcome the resulting loss of manufacturing jobs5. But in reality, new service-sector jobs have been created far more rapidly than old manufacturing jobs have disappeared. In 1960, about 35 percent of nonagricultural workers in the United States produced goods and 65 percent produced services. By 2004, only about one-sixth of the United States' nonagricultural jobs were in goods-producing industries, while five-sixths produced services. This trend is worldwide and continuing.

It's also important to point out that the years from 1960 to 2004 saw a meteoric rise in the average standard of living for the United States, on a scale that's basically unheard of in history. In fact, it was SUCH a huge rise that it became an expectation that your children would live better than you did, and the inability to keep that basic expectation in place (which has become a core part of the so-called "American Dream") that creates major societal angst on the part of the United States today.

We are now i nthe arly stages of a third Industrial Revolution--the information age. The cheap and easy flow of information around the globe has vastly expanded the scope of tradable services, and there is much more to come. Industrial revolutions are big deals. And just like the previous two, the third Industrial Revolution will require vast and usettling adjustments in the way Americans and residents of other developed countries work, live, and educate their children.

Wow, nothing unsettles people more than statements like "the world you know will cease to exist" and the like. But think about this for a second: despite the basic "growing pains" that accompanied the transitions themselves, on just about every quantifiable scale imaginable, we live a much better life today than our forebears did just two hundred years ago, and orders of magnitude better than our forebears did three hundred or more years ago (before the first Industrial Revolution). And if you still hearken back to the days of the "American farmer" with some kind of nostalgia, you never worked on a farm. Trust me on this.

So what does this mean?

But now we start to come to the interesting part of the article.

But a bit of historical perspective should help temper fears of offshoring. The first Industrial Revolution did not spell the end of agriculture, or even the end of food production, in the United States. It jus tmean that a much smaller percentage of Americans had to work on farms to feed the population. (By charming historical coincidence, the actual number of Americans working on farms today--around 2 million--is about what it was in 1810.) The main reason for this shift was not foreign trade, but soaring farm productivity. And most important, the massive movement of labor off the farms did not result in mass unemployment. Rather, it led to a large-scale reallocation of labor to factories.
Here's where we get to the "hole" in the argument. Most readers will read that paragraph, do the simple per-capita math, and conclude that thanks to soaring productivity gains in the programming industry (cite whatever technology you want here--Ruby, objects, hardware gains, it really doesn't matter what), the percentage of programmers in the country is about to fall into a black hole. After all, if we can go from 84 percent of the population involved in agriculture to less than 2% or so, thanks to that soaring productivity, why wouldn't it happen here again?

Therein lies the flaw in the argument: the amount of productivity required to achieve the desired ends is constant in the agriculture industry, yet a constantly-changing dynamic value in software. This is also known as what I will posit as the Groves-Gates Maxim: "What Andy Groves giveth, Bill Gates taketh away."

The Groves-Gates Maxim

The argument here is simple: the process of growing food is a pretty constant one: put seed in ground, wait until it comes up, then harvest the results and wait until next year to start again. Although we might have numerous tools that can help make it easier to put seeds into the ground, or harvesting the results, or even helping to increase the yield of the crop when it comes up, the basic amount of productivity required is pretty much constant. (My cousin, the FFA Farmer of the Year from some years back and a seed hybrid researcher in Iowa might disagree with me, mind you.) Compare this with the software industry: the basic differences between what's an acceptable application to our users today, compared to even ten years ago, is an order of magnitude different. Gains in productivity have not yielded the ability to build applications faster and faster, but instead have created a situation where users and managers ask more of us with each successive application.

The Groves-Gates Maxim is an example of that: every time Intel (where Andy Groves is CEO) releases new hardware that accelerates the power and potential of what the "average" computer (meaning, priced at somewhere between $1500-$2000) is capable of, it seems that Microsoft (Mr. Gates' little firm) releases a new version of Windows that sucks up that power by providing a spiffier user interface and "eye-candy" features, be they useful/important or not. In other words, the more the hardware creates possibilities, the more software is created to exploit and explore those possibilities. The additional productivity is spent not in reducing the time required to produce the thing desired (food in the case of agriculture, an operating system or other non-trivial program in the case of software), but in the expansion of the functionality of the product.

This basic fact, the Groves-Gates Maxim, is what saves us from the bloody axe of forced migration. Because what's expected of software is constantly on the same meteoric rise as what productivity gains provide us, the need for programmer time remains pretty close to constant. Now, once the desire for exponentially complicated features starts to level off, the exponentially increasing gains in productivity will have the same effect as they did in the agricultural industry, and we will start seeing a migration of programmers into other, "personal service" industries (which are hard to offshore, as opposed to "impersonal service" industries which can be easily shipped overseas).


What does this mean for programmers? For starters, as Dave Thomas has already frequently pointed out on NFJS panels, programmers need to start finding ways to make their service a "personal service" position rather than an "impersonal service" one. Blinder points out that the services industry is facing a split down the middle along this distinction, and it's not necessarily a high-paying vs low-paying divide:

Many people blithely assume that the critical labor-market distinction is, and will remain, between highly educated (or highly-skilled) people and less-educated (or less-skilled) people--doctors versus call-center operators, for example. The supposed remedy for the rich countries, accordingly, is more education and a general "upskilling" of the work force. But this view may be mistaken. Other things being equal, education and skills are, of course, good things; education yields higher returns in advanced societies, and more schooling probably makes workers more flexible and more adaptable to change. But the problem with relying on education as the remedy for potential job losses is that "other things" are not remotely close to equal. The critical divide in the future may instead be between those types are work that are easily deliverable through a wire (or via wireless connections) with little or no diminution in quality and those that are not. And this unconventional divide does not correspond well to traditional distinctions between jobs that require high levels of education and jobs that do not.

A few disparate examples will illustrate just how complex--or, rather, how untraditional--the new divide is. It is unlikely that the services of either taxi drivers or airline pilots will ever be delivered electronically over long distances. The first is a "bad job" with negligible educational requirements; the second is quite the reverse. On the other hand, typing services (a low-skill job) and security analysis (a high-skill job) are already being delivered electronically from India--albeit on a small scale so far. Most physicians need not fear that their jobs will be moved offshore, but radiologists are beginning to see this happening already. Police officers will not be replaced by electronic monitoring, but some security guards will be. Janitors and crane operators are probably immune to foreign competition; accountants and computer programmers are not. In short, the dividing line between the jobs that produce services that are suitable for electronic delivery (and are thus threatened by offshoring) and those that do not does not correspond to traditional distinctions between high-end and low-end work.

What's the implications here for somebody deep in our industry? Pay close attention to Blinder's conclusion, that computer programmers are highly vulnerable to foreign competition, based on the assumption that the product we deliver is easily transferable across electronic media. But there is hope:

There is currently not even a vocabulary, much less any systematic data, to help society come to grips with the coming labor-market reality. So here is some suggested nomenclature. Service that cannot be delivered electronically, or that are notably inferior when so delivered, have one essential characteristic: personal, face-to-face contact is either imperative or highly desirable. Think of hte waiter who serves you dinner, the doctor you gives you your annual physical, or the cop on the beat. Now think of any of those tasks being performed by robots controlled from India--not quite the same. But such face-to-face human contact is not necessary in the relationship you have with the telephone operator who arranges your conference call or the clerk who takes your airline reservation over the phone. He or she may be in India already.

The first group of tasks can be called personally-delivered services, or simply personal services, and the second group of impersonally delivered services, or impersonal services. In the brave new world of globalized electronic commerce, impersonal services have more in common with manufactured goods that can be put in boxes than they do with personal services. Thus, many impersonal services are destined to become tradable and therefore vulnerable to offshoring. By contrast, most personal services have attributes that cannot be transmitted through a wire. Some require face-to-face contact (child care), some are inherently "high-risk" (nursing), some involve high levels of personal trust (psychotherapy), and some depend on location-specific attributes (lobbying).

In other words, programmers that want to remain less vulnerable to foreign competition need to find ways to stress the personal, face-to-face contact between themselves and their clients, regardless of whether you are a full-time employee of a company, a contractor, or a consultant (or part of a team of consultants) working on a project for a client. Look for ways to maximize the four cardinalities he points out:
  • Face-to-face contact. Agile methodologies demand that customers be easily accessible in order to answer questions regarding implementation decisions or to resolve lack of understanding of the requirements. Instead of demanding customers be present at your site, you may find yourself in a better position if you put yourself close to your customers.
  • "High-risk". This is a bit harder to do with software projects--either the project is inherently high-risk in its makeup (perhaps this is a mission-critical app that the company depends on, such as the e-commerce portal for an online e-tailer), or it's not. There's not much you can do to change this, unless you are politically savvy enough to "sell" your project to a group that would make it mission-critical.
  • High levels of personal trust. This is actually easier than you might think--trust in this case refers not to the privileged nature of therapist-patient communication, but in the credibility the organization has in you to carry out the work required. One way to build this trust is to understand the business domain of the client, rather than remaining aloof and "staying focused on the technology". This trust-based approach is already present in a variety of forms outside our industry--regardless of the statistical ratings that might be available, most people find that they have a favorite auto repair mechanic or shop not for any quantitatively-measurable reason, but beceause the mechanic "understands" them somehow. The best customer-service shops understand this, and have done so for years. The restaurant that recognizes me as a regular after just a few visits and has my Diet Coke ready for me at my favorite table is far likelier to get my business on a regular basis than the one that never does. Learn your customers, learn their concerns, learn their business model and business plan, and get yourself into the habit of trying to predict what they might need next--not so you can build it already, but so that you can demonstrate to them that you understand them, and by extension, their needs.
  • Location-specific attributes. Sometimes, the software being built is localized to a particular geographic area, and simply being in that same area can yield significant benefits, particularly when heroic efforts are called for. (It's very hard to flip the reset switch on a server in Indiana from a console in India, for example.)
In general, what you're looking to do is demonstrate how your value to the company arises out of more than just your technical skill, but also some other qualities that you can provide in better and more valuable form than somebody in India (or China, or Brazil, or across the country for that matter, wherever the offshoring goes). It's not a guarantee that you might still be offshored--some management folks will just see bottom-line dollars and not recognize the intangible value-add that high levels of personal trust or locality provides--but it'll minimize it on the whole.

But even if this analysis doesn't make you feel a little more comfortable, consider this: there are 1 billion people in China alone, and close to the same in India. Instead of seeing them as potential competition, imagine what happens when the wages from the offshored jobs start to create a demand for goods and services in those countries--if you think the software market in the U.S. was hot a decade ago, where only a half-billion (across both the U.S. and Europe) people were demanding software, now think about it when four times that many start looking for it.


1 Which in of itself is an interesting statistic--it implies that offshoring is far less prevalent than some of people worried about it believe it to be, including me.

2 Interesting bit of trivia--part of the reason that advantage shifted was because the US stole (yes, stole, as in industrial espionage, one of the first recorded cases of modern industrial espionage) the plans for modern textile machinery from the UK. Remember that, next time you get upset at China's rather loose grip of intellectual property law....

3 Which, by the way, was a large part of the reason we fought the Civil War (the "War Between the States" to some, or the "War of Northern Aggression" to others)--the Carolinas depended on slave labor to pick their cotton cheaply, and couldn't acquire Northern-made machinery cheaply to replace the slaves. Hence, for that (and a lot of other reasons), war followed.

4 An interesting argument--is there any real difference between transportation and communications? One ships "stuff", the other "data", beyond that, is there any difference?

5 And, I'd like to point out, the shrinking environmental damage that can arise from a manufacturing-based economy. Services rarely generate pollution, which is part of the clash between the industrialized "Western" nations and the developing "Southern" ones over environmental issues.


"Offshoring: The Next Industrial Revolutoin?", by Alan S. Blinder, Foreign Affairs (March/April 2006), pp 113 - 128.

Tech Predictions: 2007 Edition

So, in what's become an ongoing tradition, this is the time of year when I peer into the patented Ted Neward Crystal Ball (TM) (operators are standing by!), see what it tells me about technology trends and ideas for the coming year, and report them to you. The usual disclaimers apply, meaning I'm not getting any sort of endorsement deals to mention anybody's technology here, I'm not speaking for anybody but myself in this, and so on. And, in order to prove that I'm not an analyst group like Forrester or Burton or any of those other yahoos, in a separate post, I'll look over my predictions for 2006 and see how they panned out, thus proving that the patented Ted Neward Crystal Ball (TM) is just as capable of mistakes as any other crystal ball of course, right all the time.

2006 was an interesting year, in that a lot of interesting things happened this year for developers. For the .NET crowd, Visual Studio 2005 and SQL Server 2005 finally became widely available to them (yes, it shipped in 2005 but it took a bit for it to percolate through the community), and NetFX 3 (aka .NET 3.0, aka Indigo/Avalon/Workflow) shipped in Q4, not to mention Vista itself, meaning there was all kinds of new stuff to play with. For the Java crowd, Spring 2.0 shipped, Geronimo 1.0 shipped, and Sun decided to finally open the doors on the JDK (apparently not realizing that a lot of us had already slipped in the back way through the doors marked "SCSL license" and "JRL license" since JDK 1.2...). Meanwhile, Ruby continued to amaze those who'd never seen a dynamic/scripting language before, and Rails continued to amaze developers who'd never seen a VB demo before. More WS-* specs shipped, people started talking about JavaScript Object Notation (JSON), RSS/Atom continued to draw attention in droves, and marketing guys looked for all kinds of places they could hang the Tim O'Reilly-inspired "Web 2.0" meme anywhere they could. And yet, through it all, developers somehow ignored the noise and kept working.

Without further ado...

  • General: Analysts will call 2007 the Year of the {Something}, where I bet that {Something} will be either "ESB" or "SOA". They will predict that companies adopting {Something} will save millions, if not billions, if only they rush to implement it now. They will tag this with a probability of .8 in order to CYA in case {Something} doesn't pan out. (Yes, I've read far too many of these reports--I'm personally convinced that each of the analyst companies has a template buried away in their basement that they pull out each time they need a new one, and they just do a global search-and-replace of "{Something}" with whatever the technology du jour happens to be.)
  • .NET: Thousands of developers will horribly abuse WPF in ways that can only be called nightmarish, thus once again proving the old adage that "just because you can doesn't mean you should" still holds. WPF's capabilities with video will prove, in many ways, to be the modern equivalent to the "blink" tag in HTML. This will provide some author with a golden opportunity: "WPF Applications That Suck". Alan Cooper will re-release "About Face", updated to include WPF UI elements.
  • .NET: Thousands of developers will look to Redmond for an answer to the question, "Which should I use? BizTalk, Windows Workflow, or SQL Server Service Broker?", and get no clear answer.
  • Windows: Microsoft will try, once again, to kill off the abomination that was called the Windows 95/98/Me line of operating systems, and will once again have to back off as industry outcries of protest (on behalf of little old ladies who are the only ones left running Windows 95/98/Me and probably haven't turned their machine on in months, at least not since the grandkids last visited) go ballistic.
  • Windows: Ditto for Visual Basic 6.0, except now the outcry will be on behalf of developers who aren't capable of learning anything new. Sun will use the resulting PR to announce Project YAVKRWMITT (Yet Another VB Killer Really We Mean It This Time, pronounced "YAV-kermit") on java.net. Meanwhile, efforts to make CLASSPATH into something a VB 6 guy actually has a prayer of understanding will go quietly ignored.
  • Java: JSR 277 will continue to churn along, and once the next draft ships, publicly nobody will like what we produce, though quietly everybody will admit it's a far cry better than what we have now, and when it ships in JDK 7 will be adopted widely and quietly.
  • Java: Thousands of new ideas and proposals to extend the Java language in various ways will flood into the community, now that developers can start hacking on it for themselves thanks to the OpenJDK. Only a small fraction of these will ever get beyond the concept stage, and maybe one or two will actually be finished and released to the Web for consideration by the community and the JCP. Thousands more Java developers craving Alpha-Geek status will stick a "Hello, world" message into the compiler's startup sequence, then claim "experienced with modifying the OpenJDK Java compiler" on their resume and roundly criticize Java in one way or another by saying, "Well, I've looked at the code, and let me tell you....".
  • .NET: Somewhere, a developer will realize that SQL Server 2005 can be a SOAP/WSDL XML service endpoint, and open it up as a private back-channel for his application to communicate with the database through the firewall "for performance reasons" (meaning, "So I can avoid having to talk to the app server in between my web server and my database"). With any luck, the DBA will kill him and hide the body before anybody can find and exploit it.
  • General: Yet Another Virus That's Microsoft's Fault will rip through the Internet, and nobody will notice that the machines affected are the ones that aren't routinely administered or receive updates/patches. Companies will threaten Microsoft with million-dollar lawsuits, yet will fire none of their system administrators who lovingly lavish whole days tuning their Linux IRC servers yet leave the Windows Exchange Server still running Windows NT 4.0.
  • General: Interest in JSON will escalate wildly, hyped as the "natural replacement for XML" in building browser-to-server connections, owing to its incredible simplicity in expressing "object" data. Folks, JSON is a useful format, but it's not a replacement for XML (nor is XML a replacement for it, either). What made XML so popular was not is hierarchical format (Lord above, that's probably the worst part of it, from where we as developers sit), nor its HTML-like simplified-SGML syntax. What made XML interesting was the fact that everybody lined up behind it--Microsoft, Sun, BEA, Oracle, IBM, there's not a big vendor that didn't express its undying love and devotion to XML. I sincerely doubt JSON will get that kind of rallying effect. (And if you're going to stand there and suggest that JSON is better because its simpler and therefore more approachable for developers to build support for themselves, quite honestly, I thought we were trying to get out of developers building all this communications infrastructure--isn't that what the app servers and such taught us?)
  • General: Interest in Java/.NET interopability will rise as companies start to realize that (a) the WS-* "silver bullet" isn't, (b) ESB, XML, and SOA are just acronyms and won't, in of themselves, solve all the integration problems, and (c) we have lots of code in both Java and .NET that need to talk to each other. This may be a self-serving prediction, but I got a LOT of interest towards the end of this year in the subject, so I'm guessing that this is going to only get bigger as the WS-* hype continues to lose its shine in the coming years.
  • Ruby: Interest in Java/Ruby and .NET/Ruby interoperability is going to start quietly making its presence felt, as people start trying to wire up their quick-to-write "stovepipe" RAILS apps against other systems in their production data center, and find that Ruby really is a platform of its own. RubyCLR or JRuby may be part of the answer here, but there's likely some hidden mines there we haven't seen yet.
  • Languages: A new meme will get started: "JavaScript was that thing, that little toy language, that you used to do stuff in the HTML browser. ECMAScript, on the other hand, is a powerful and flexible dynamic programming language suitable for use in all sorts of situations." Pass it on. If you get it, don't tell anybody else. (Don't laugh--it worked for "The Crying Game".) It's the only way JavaScript ECMAScript will gain widespread acceptance and shed the "toy" label that JavaScript has.
  • Languages: Interest in functional-object hybrid languages will grow. Scala, Jaskell, F#, and others not-yet-invented will start to capture developers' attention, particularly when they hear the part about functional languages being easier to use in multi-core systems because it encourages immutable objects and discourages side effects (meaning we don't have to worry nearly so much about writing thread-safe code).
  • Languages: Interest in Domain-specific languages will reach a peak this year, but a small backlash will begin next year. Meanwhile, more and more developers will realize that one man's "DSL" is another man's "little language", something UNIX has been doing since the early 70's. This will immediately take the shine off of DSLs, since anything that we did in the 70's must be bad, somehow. (Remember disco?)
  • General: Rails will continue to draw developers who want quick-fix solutions/technologies, and largely that community will ignore the underlying power of Ruby itself. The draw will start to die down once Rails-esque feature ideas get folded into Java toolkits. (Rails will largely be a non-issue with the .NET community, owing to the high-productivity nature of the drag-and-drop interface in Visual Studio.)
  • Java: Interface21 is going to start looking like a "big vendor" alongside BEA and IBM. I was talking with some of the I21 folks in Aarhus, Denmark at JAOO, and one of them casually mentioned that they were looking at a Spring 2.1 release somewhere in mid-2008. Clearly Spring is settling into eighteen-month major-version release cycles like all the big (meaning popular), established software systems have a tendency to do. This is both a good thing and a bad thing--it's good in that it means that Spring is now becoming an established part of the Java landscape and thus more acceptable to use in production environments, but it's bad in that Spring is now going to face the inevitable problem all big vendors face: trying to be all things to all people. This is dangerous, both for Interface21 and the people relying on Spring, largely because it means that Spring faces a very real future of greater complexity (and there are those, myself included, who believe that Spring is too complex already, easily on par with the complexity seen in EJB, POJOs notwithstanding).
  • General: Marc Fleury will get a golden parachute from Red Hat (at their request and to their immense relief), and hopefully will retire to his own small island (might I suggest Elba, la petite corporal?) to quietly enjoy his millions. A shame that the people who did most of the real work on JBoss won't see a commensurate reward, but that's the way the business world works, I guess.
  • General: Some company will get millions to build an enterprise product on the backs of RSS and/or Atom, thus proving that VCs are just as stupid and just as vulnerable to hype now as they were back in the DotCom era.
  • General: Somebody will attempt to use the phrase "Web 2.0" in a serious discussion, and I will be forced to kill them for attempting to use a vague term in a vain effort to sound intelligent.
  • Web clients: Ajax will start to lose its luster when developers realize the power of Google Maps isn't in Ajax, but in the fact that it's got some seriously cool graphics and maps. (Or, put another way, when developers realize that Ajax alone won't make their apps as cool as Google Maps, that's it's the same old DHTML from 1998, the hype will start to die down.)
  • XML: Somebody, somewhere, will realize that REST != HTTP. He will be roundly criticized by hordes of HTTP zealots, and quietly crawl away to go build simpler and more robust systems that use transports other than HTTP.
  • XML: Somebody, somewhere, will read the SOAP 1.2 specification. H.P. Lovecraft once suggested, loosely paraphrased, the the day Man understands the nature of the universe, he will either be driven into gibbering insanity, or flee back into ignorance in self-preservation. Ditto for the day Man reads the SOAP 1.2 spec and realizes that SOAP is, in fact, RESTful.
  • Security: The US Government will continue its unbelievable quest to waste money on "security" by engaging in yet more perimeter security around airports and other indefensible locations, thus proving that none of them have bothered to read Schneier and learn that real security is a three-part tuple: prevention, detection, and response.
  • Security: Thousands of companies will follow in the US Government's footsteps by doing exactly the same thing. (Folks, you can't solve all your problems with cryptography, no matter how big the key size--you just end up with the basic problem of where to store the keys, and no, burying them inside the code isn't going to hide them effectively.)
  • Security: More and more rootkits-shipping-with-a-product will be discovered. We used to call it "getting close to the metal", now it's a "rootkit". With great power comes great responsibility... and, as many consumers have already discovered, with great power also comes a tendency to create greater instability...
  • General: Parrot will ship a 1.0 release. Oh, wait, hang on, sorry, I bumped into the crystal ball and accidentally set it to 2017.
  • .NET: Microsoft will ship Orcas (NetFX 3.5). (Sorry, crystal ball's still set on 2017. Trying to fix it...)
  • .NET: Vista will surpass Windows XP in market penetration. (Let's see, almost got it set back to 2007, bear with me... There. Got it.)
  • General: I will blog more than I did this year. (Hell, I couldn't blog less, even if I tried.)
  • General: Pragmatic XML Services, Pragmatic .NET Project Automation and Effective .NET will ship. (Wait, is the crystal ball still on 2017...?)

Warning: XSS attack in PDF URLs

Just heard this through the OWASP mailing list, and it's a dandy:

I wanted to give everyone all a heads-up on a very serious new application security vulnerability that probably affects you. Basically, any application that serves PDF files is likely to be vulnerable to XSS attacks.

Attackers simply have to add an anchor containing a script, e.g. add #blah=javascript:alert(document.cookie); to ANY URL that ends in .pdf (or streams a PDF). The browser hands off the anchor to the Adobe reader plugin, and the script then runs in the victim’s browser.

You can find more information here: http://www.gnucitizen.org/blog/universal-pdf-xss-after-party/

You can protect yourself by upgrading your browser and Adobe Reader. There are many vulnerable browser/plugin combinations in use, including Firefox. However, IE7 and IE6 SP2 do not appear vulnerable.

Protecting the users of your application from attack is more difficult. This problem is entirely in the browser and the Adobe reader. The anchor is not even passed from the browser to the web application, so there’s really not much you can do in your code to detect an attack. You could stop serving PDF documents or move them to a different server, but that’s not realistic for many organizations.

Jeff Williams, Chair, The OWASP Foundation

Now, a couple of thoughts come to mind:

1. First and foremost, if your application serves PDFs, make sure your clients know to upgrade to the latest Acrobat version, since that seems (based on how I read the above) to be protected against the XSS attak; if it's not, though, Adobe will fix it soon (I would hope, anyway), and thus you'll be back to making sure your clients know to upgrade to the latest Acrobat version.
2. Secondly, this is technology-agnostic, so regardless of your platform (Java, .NET or Rails), you're vulnerable. (Such is always the case with XSS attacks.)
3. How many developers will actually take steps to try and prevent it (such as, for example, ensuring that PDF URLS received aren't trailing any fragments before sending the URL request on for Adobe to process)?
4. How long before somebody figures out a way to make this all Microsoft's fault? Will this gather any press coverage, and if it does, will they note that IE 6 SP2 and IE 7 don't seem to be affected by the attack? Will Slashdot even bother with a footnote? (My best guess would be, 1 week, yes, no, and no, respectively.)

Original Post

LINQ: the January CTP is out.

The January CTP for the next version of Visual Studio, code-named "Orcas" is now available. By CTP, Microsoft implies a Community Technology Preview - it is essentially a lightweight beta ship vehicle for a future product. By shipping this preview, Microsoft is trying to solicit feedback on the feature set ("I really like having that new dial to adjust my frimble."), on the realization of those features ("I like having that lever, but when I flip it this other thingy comes loose."), and on the prioritization of features ("The new dial makes my life easier, but I don't find much use for that new lever you've added.") The main highlights of this CTP are the new ADO.NET stuff (including the Entity Data Model) and the latest LINQ bits.

You might ask, just what is this LINQ thing? LINQ, the name, derives from "Language Integrated Query." LINQ is a codename for a set of extensions to the .NET Framework that encompass language-integrated query, set, and transform operations. It extends C# and Visual Basic with native language syntax for queries and provides class libraries to take advantage of these capabilities.

I took that description right from the main LINQ page on MSDN. Let me decompose that a little, because it's sort of dense. It makes sense if you already know what LINQ is, but if you come into it without any prior knowledge, I'm not so sure it clarifies anything.

First, LINQ is a set of extensions to a couple .NET languages, as well as to the base framework (or base class library). It's important to realize that LINQ is not just a new class library. LINQ also implies language syntax changes. You might be a skeptic and say, languages schmanguages - they're all Turing-complete, right? But no one who looks at Perl, T-SQL, and C# could avoid the conclusion that different languages have different balance points - they are optimized for different things. What LINQ is doing is extending the existing language syntax that we know and love in C# and VB today, to add some new stuff.

New stuff to do what? Ahh, to do query, set and transform operations. A query, like "Give me all the customers who ordered $10000 worth of product last month." Today, querying data is done differently, depending on where you get your data. If your data is in a relational database, then you use the DBMS query engine to do the queries. You write some SQL code as a string and send it to the database and it gives you rows of data back. (if you are an EJB programmer, then you write an EJBQL string, and a findByXxx method and a bunch of other stuff. ) On the other hand if you have your data in an XML document, then you can perform XPath or XQuery on it. Finally if you have your data in a plain old in-memory collection of objects, you may have to manually iterate through the collection to extract the data items that match your constraints. foreach anyone?

What LINQ attempts to do is standardize the way these queries are performed, regardless of the datasource, and secondly, to add query as a language feature, so that developers can get all the benefits (intellisense, and so on) of that.

For years different teams and projects in the software industry have grappled with the problem of object-relational mapping. The whole point was to save time in development. We all realize that much (most?) of the world's transactional data is stored in relational stores, and most new app development is done in object-oriented languages. So the challenge we've been grappling with has been, how to map the relational data to an in-memory object state, in effect making it simpler and easier for an OO programmer to fiddle with data. This effort has seen some small successes, but also some large failures. Also as more and more data moves into XML, mapping between relational stores and in-memory object representations seems to be missing half the point. We need something different!

Microsoft is taking a different tack on the question of how best to improve developer productivity around data handling, given the two constraints above - that data is relational (but more and more XML) and code is OO. With LINQ, Microsoft is saying, we think the opportunity for improvement is in the ability to query data more simply. It's not so much in the mapping - there are solutions here that get people "most of the way there". The challenge now is, once you mapped your DBMS schema to your Object schema, once you have filled a collection of objects from your database, then what? Can you filter that collection ("Give me all the customers that ordered over $10000 in the last month. Ok, now give me the ones whose orders included this particular product. Wait, now let me see the ones that ordered this other product.")

Or think a little more generally and consider an object that aggregates data from multiple stores - maybe some of them relational and some not. And you'd like to do "join" queries across all of that data. Wouldn't it be nice if that were possible without requiring the developer to speak 3 different dialects of query language - T-SQL, XPath and something else?

So that's what LINQ represents - Microsoft's integration of this query capability into the programming language itself. It is worth checking out; it is one of the things Microsoft believes will distinguish the next version of .NET from other alternatives, like Java. A big innovation. It seems that at least some people outside of Microsoft tend to agree. And with the January CTP of Visual Studio "Orcas", now you can check it out.

What about Interop, you say? Well that brings me to my point. Currently the design of LINQ allows for extensions so that one could integrate this in-language query capability with any backing store. Microsoft will do the work to integrate SQL Server, as well as XML content with the capability. We're hoping that given enough interest, we'll see a clear justification for doing integration with other relational stores. If you have strong opinions on this, please let me know.


Original Post

C# 3.0 Feature Notes

Anders Heljsberg has been giving us an overview of new C# 3.0 features this morning. I’ve made a few notes. Most of this info is available elsewhere, and I’m not re-typing the code from his demos, but there might be some useful tidbits here for some folks.

First by way of background, Anders noted the evolution of C# and the key feature that defined each version:

C# 1.0 – Managed Code
C# 2.0 – Generics
C# 3.0 – Language Integrated Query (LINQ)

Some C# 3.0 Design Goals (partial list):

· Integrate objects, data, xml

· Increase concision of the language

· Add functional programming constructs

Extension Methods

Allow adding methods to any classes that have a particular signature. In fact, it is simply a compiler-enabled illusion that simplifies calling static methods. For example, if you have a static method like this one:

public static string Concatenate(IEnumerable strings, string separator) {…}

You could call it like so:
string result = SomeClass.Concatenate(myListOfStrings, mySeparator);

Using Extension methods you would alter your Concatenate method like so:

public static string Concatenate(this IEnumerable strings, string separator) {…}

This static method must reside in a static class. It is brought into scope by using the using Namespace; syntax. Once its namespace is in scope, the above code could be rewritten as:

string result = myListOfStrings.Concatenate(mySeparator);

The way this works is the compiler will look to see if myListOfStrings actually has a method of its own called Concatenate. If it doesn’t find one, then it will search through all of the static classes in scope to see if any apply to that type (in this case IEnumerable). If so, then it will essentially rewrite the code into the static method call that passes the object as the first parameter to the static method.

One very cool thing about these, which may not be immediately obvious, is that they can reference a wide variety of types. If the first parameter were object, for instance, then it could be applied to any type in the framework (not that this would necessarily be wise…).


LINQ for SQL does not support SQL Cache Invalidation, unfortunately. SQL Cache Invalidation against SQL 2005 is currenty done in ADO.NET using the SqlCommand object, which no doubt is used somewhere along the line as LINQ communicates with the database. It would be cool if there were a way to inform LINQ that this query should be cached until a notification is received from the database that the resultset has changed, since this can provide immense performance benefits.

On the plus side, LINQ will ship with a dev toolkit for writing data providers, which would if nothing else let me hack the default SQL behavior to support SQL Cache Invalidation. It’s been my experience that ORM tools do not care to consider caching of resultsets, and in this LINQ for SQL is no exception.


Interoperability: Bridging .NET and Java

Web Services
Distributed Objects
Business Process Integration

In the heterogeneous world of enterprise computing, interoperability is no longer a luxury but a necessity. The multiple tier architecture of modern enterprises coupled with multiple development tools can lead to multiple headaches for developers. So what's a poor developer to do? Fortunately (some would say unfortunately), there are many ways to tackle this problem.

The resources I've assembled below should provide you with a broad overview of some of the techniques being used to bridge the worlds of .NET and Java technology. We'd love to hear about the interoperability issues you've encountered and how you went about solving them.


Interoperability between Java EE technology and .NET applications

Dr. Dobb's "Interoperability and Integration" poster (PDF)

Webcast: Application interoperability with Microsoft .NET and J2EE

Top ten tips for Web services interoperability

Practical .NET-to-Java interop solutions

Application interoperability: Microsoft .NET and J2EE


Advanced message queue protocol to commoditize messaging

How to do MTOM interop

Simple JAVA and .NET SOA interoperability

Java and .NET interop via MSMQ

.NET integration with WebSphere Queues

IBM Message Service Client for .NET

Web Services

Web Services Interoperability Technology (WSIT)

An overview of Sun's Project Tango

WS-I focusing on making Web services enterprise-class

Web Services for .NET and J2EE interoperability

Integrating .NET Web services with Java and classic ASP clients for interoperability

WebSphere and .Net interoperability using Web services

Web services programming (with passion!)

Real interoperability between J2EE and .NET Web services: Practical advice for developers

Distributed Objects


Java and CORBA: Uniting ubiquity and interoperability

An overview of RMI applications

Introduction to Java RMI

The new RMI

.NET Remoting overview

.NET Remoting

Business Process Integration

BPEL Learning Guide

BPM Learning Guide

BPEL and Java

Vista Series: Windows Workflow Foundation

Windows Workflow Foundation: An open model

Simple Human Workflow with Windows Workflow Foundation

Java Technology: Project Open ESB

JBI and service-oriented integration

Why Java Business Integration

Enterprise .NET Community: XAML: Common ground for developers, designers

By Colleen Frye

XAML - Extensible Application Markup Language (pronounced "Zammel") - is fundamental to Microsoft Windows Vista and .NET 3.0, and promises to bridge the gap between designers and developers. To use these new tools effectively, though, organizations need to re-examine their software engineering practices and strengthen their software development life cycle (SDLC), according to Burton Group analyst Chris Howard in a recent webcast on XAML.

"It opens the door for non-engineer developers," said Howard, vice president and service director for application platform strategies at Midvale, Utah-based Burton Group. "People all across the SDLC are starting to contribute, so it places more stress on the SDLC."

XAML is a declarative programming language that provides the grammar for domain-specific languages (DSLs), and underlies both Windows Presentation Foundation (WPF) and Windows Workflow Foundation (WF).

According to Howard, XAML cleanly separates the roles in a project while at the same time increasing the compatibility of the artifacts produced, since both user interface designers and developers are speaking the same "language." "The promise of DSLs is providing people with the right tool at the right time," Howard said.

Declarative programming increases the level of abstraction, which increases developer productivity and is a "hallmark" of the evolution of technology, Howard said, but there are some caveats. "You're working with an environment that architecturally keeps you safe, but the complaint I hear is maybe it's too constraining," Howard said.

"Some developers will love it, some will want to keep their distance at first," said Rob Relyea, lead program manager for the WPF Team at Microsoft.

A solution, Howard said, is to combine the declarative aspects with code behind, but organizations need to think through their designs. "By abstracting, you're making it more productive for the things you don't want to write more than once, but it allows you to hook into code behind. So you can do pieces that are declarative, pieces that are structured, pieces as Web services. But it means you have to consider design very carefully. What am I going to make declarative? And you have to build those standards into the way you build applications," Howard said.

"In WPF and Workflow, both models give the option of ignoring markup completely and coding away, and both allow the flexibility of XAML parsed at runtime," Relyea said. "The story of XAML started on the WPF team, and over time we saw WF with a similar technology." The teams talked about leveraging XAML for WF as well, he said. "XAML encoding of workflow is clearly taking off. They're [the WF team] finding many of their customers like to have it human readable/writable for quick manipulation."

For development teams, XAML is the unifier in what has often been a disjointed process, Howard said. "The SDLC today in most shops, you have a user experience team working in Visio, you've got a marketing team that produces ad materials, you've got business analysts building workflow in some kind of tool. There is a lot of disjunction. XAML as a universal grammar unifies," Howard said. "If the idea works - a unifying language across artifacts - it becomes a powerful thing, but it causes you to revisit [the SDLC]."

XAML is the language used in Microsoft's new Expression line of tools, which are targeted to designers. Now, the output from the Expression tools can be the input to Visual Studio. This commonality allows domain experts to contribute directly to development, without losing anything in translation, Howard said.

With this ability, a big question development organizations need to ask, Howard said, is are they correctly staffed? "Most big shops I deal with, their shops are spread out and it's hard to get people on the same floor to work together, much less crossing boundaries, especially with designers," Howard said.

That's a "big challenge," Relyea agreed. "Many people don't have graphic designers along with their staff, but you've got a great story when you have a team with multiple disciplines."

In terms of staffing, an emerging area to watch is the development of user experience teams, Howard said. "They have nontraditional backgrounds, but they're experts in how people interact with machines and with each other. They can translate that into design and hand it off to the engineering process." Howard said there are examples of this approach in the financial services industry today.

However, Howard does caution that organizations need to clarify the roles of developers and designers. He also said there are some security concerns with the ability to pull XAML in from the server. "You like to think people have things locked down, but there is still a potential risk," he said.

Still, Howard recommends that now is a good time to get started with .NET 3.0. He suggests staffing development organizations carefully, "and don't limit yourself just to technology roles."

From Microsoft, Howard said he is looking for "advice on where to write what. Organizations will need guidance in terms of best practices."

Colleen Frye is a contributing writer for TheServerSide.NET
Original Posting

Source Safe Web Interface v2.0 Released

SSWI is a web based addon to Visual Source Safe that allows you to use all of the common source safe features from your favourite web browser.

SSWI also integrates with Visual Studio to enable remote access to your corporate VSS database from anywhere in the world as if you were still in the office.

It provides remote access to all the features of Visual Source Safe without the need to install software on the end user PC. SSWI is a fully web based system, that will work on any PC or MAC (OSX) with IE6+, Firefox or Safari and the Java runtime

  • Fast: Using Source Safe Web Interface can be up to 12 times faster than using VSS over a VPN or wide area network.
  • Secure: All data is encrypted so you can be sure your code is secure.
  • User Friendly: The SSWI user interface is almost identical to the actual SourceSafe client, which means your users will have no problem migrating to SSWI.
  • Cross Platform: SSWI is designed to work on Windows, Mac (OSX) and Linux.
  • Easy to Install: SSWI is packaged in a single installer and includes a Server Manager application that enables you to quickly install and configure SSWI, without the pain of configuration files and database access.
  • Minimal Client Side Requirements: SSWI is a browser based application that requires only a modern browser and the Java runtime. In most cases you will find that the end user machine already contains these prerequisites.
  • IDE Integration: Using the SSWI IDE integration you can access remote VSS databases from Visual Studio 6, 2002, 2003, 2005 and any other IDE that supports the MSSCCI interface (including Delphi 2005).
  • Cross Browser: SSWI supports Internet Explorer 6+, Firefox 1+ and Safari (Mac OSX).
  • Developers API: Using the SSWI developer API you can build source safe operations into any application or process simply by calling an XML web service.

C# godfather Anders Hejlsberg on dynamic languages, LINQ

As C# continues to gain adherents, developers continue to closely watch the comments of Microsoft's Anders Hejlsberg, chief designer of C#. What better way is there to gauge the direction the language will take next?

Last month at TechEd Europe in Barcelona, Hejlsberg discussed software futures that include more dynamic language support, and more integrated query support.

He suggested that a mixture of dynamic and static language traits was emerging as a prominent programming trend. Also, he and said he saw indications that Microsoft's Language Integrated Query project would be supported by third-party makers of object/relational mapping tools

Dynamic range
While static-typed object-oriented languages like C# and Java made great headway in the last five years, dynamic languages have not take an backseat. Certainly, dynamic Python has continued to flourish and Ruby has gained considerable momentum. Melding of dynamic and static alternatives are being pursued by both Java-proponent Sun and .NET-originator Microsoft.

''It is funny to watch the pendulum swing back and forth between static and dynamic typing. I think the mega trend probably is that there is more of a fusing of a number of different programming languages occurring right now,'' said Hejlsberg.

He continued: ''You can look at classical object-oriented dynamic and functional programming languages, and in a sense, they are all fusing, coming together in sort of a new style of programming language. I certainly aim for C# to stay on the forefront of that.''

Hejlsberg indicated that aspects of functional programming are creeping into the mix that makes the modern software language.

''When you ask why people like dynamic language, people often say it's because [they] have to write less or because it’s terser or succinct and you try and dig a little bit…and often one of the reasons that gets cited is there are no types there, and therefore types get in the way. We all know that strong typing is a sort of a lever. And the further you dial it up the more painful it gets.''

''However, one of the things that we are learning a lot from functional programming languages is in the area of type inference. It is one of those … 'have your cake and eat it too' kind of things. It is actually strongly typed''

''When types are gone, there are just a lot of things a tool can no longer do for you,'' Hejlsberg advised, adding that his approach was to look for ''the happy medium.''

LINQ looms
Perhaps the biggest change due in C# next year is LINQ, based on a project that Hejlsberg and others spearheaded, and which is due to ship with Orcas, the next version of Visual Studio. LINQ (for Language Integrated Query) is intended to integrate query building into the language to make it easier for developers to create data queries that work across objects, databases and XML data.

Like many things Microsoft is pursuing these days, LINQ is intended to be extensible, so it can be used across a variety of languages other than C#, not the least of which is Visual Basic.

In his video interview at TechEd Europe, Hejlsberg said he welcomes third-parties that build on the LINQ concept. ''I have already seen several ORM providers support it in their Object/Relational mappings, which is wonderful.'' A ‘Virtual’ version of that broadcast is available on the Web.

Related interview with Microsoft’s Anders Hejlsberg

Monday, January 15, 2007

Tango in Web services: Java and .NET 3.0 interoperability

Written by Arun Gupta, Sun Microsystems

WIST, for Web Services Interoperability Technology (a.k.a. Project Tango), is a Sun Microsystems initiative to provide an open source implementation of enterprise Web services technologies for interoperability between Java Web services and Microsoft Windows Communication Foundation (WCF, a.k.a Indigo).

In Web services terms, this would mean that a .NET client can invoke a Web service on Java platform, and vice versa, and still perform the intended business logic. All this needs to be done without the client and service developers worrying about how the HTTP headers are going to be interpreted, whether an image attachment sent by one stack will be received correctly by the other, and if the two stacks are agree on on-the-wire representations for secure reliable sessions.

W3C Web Services Architecture Working Group defines Web service as: A Web service is a software system designed to support interoperable machine-to-machine interaction over a network.

As is evident from the definition, the requester and provider of the Web service must be able to talk in an interoperable manner. In this blog, I’ll talk about Web Services Interoperability Technology (WSIT, pronounced same as “visit”, a.k.a. Project Tango), a Sun Microsystems initiative, an open source implementation of enterprise Web services technologies that deliver interoperability between Java Web services and Microsoft Windows Communication Found (WCF, a.k.a Indigo and available as part of .NET 3.0 platform) to help you build, deploy and maintain Composite Applications for your Service Oriented Architecture.

As an extension to JAX-WS, WSIT has four main categories, covering the key WS-* specifications:

  • Messaging (SOAP, MTOM, WS-Addressing)
  • Metadata (WSDL, WS-Policy, WS-Metadata Exchange)
  • Security (WS-Security Policy, WS-Security, WS-Trust, WS-Secure Conversation)
  • Quality-of-Service (WS-Reliable Messaging, WS-Coordination, WS-Atomic Transactions)

These are the features required by an enterprise developer to create a composite application. Apart from providing an implementation of the key WS-* specs, the distinct feature of WSIT is the letter “I” in WSIT, which stands for interoperability. The literal meaning of the word “interoperability” is: “ability of a system to work with or use the parts or equipment of another system”.

The definition is clear but the key point to highlight is that this definition assumes the two systems “just work”. And the definition does not mention any workarounds to be done by the user of the systems. The fact that two systems are working together seamlessly is completely transparent to the user. In Web services terms, this would mean that a .NET client can invoke a Web service on Java platform, and vice versa, and still perform the intended business logic. All this needs to be done without the client and service developers worrying about how the HTTP headers are going to be interpreted, whether an image attachment sent by one stack will be received correctly by the other, and if the two stacks are agree on on-the-wire representations for secure reliable sessions.

The WSIT implementation, available as part of GlassFish v2 builds, has been extensively tested to work with .NET 3.0 framework in an interoperable manner. The WSIT has successfully participated in multiple interop plug-fests where successive interoperability has been achieved. That means, in the initial plug-fests the focus was more on the basic specifications like MTOM and WS-Security. In subsequent one, more composite scenarios like secure reliable messaging and secure transactional scenarios are tested. In an interoperability plug-fest, held at Microsoft campus, in September, Sun achieved interoperability on all the simple and composite scenarios. Read more about the plug-fests here.

So if you are developing an enterprise application then WSIT not only provides you all the building blocks, with complete tooling support in NetBeans IDE 5.5 and deployment platform in GlassFish v2, but also provides an out-of-the-box interoperability with .NET framework 3.0.

So, next time you think about Java and .NET Web services interoperability, think, visit, and downloa d WSIT. This is one platform that allows you to develop secure, reliable, transactional and interoperable Web services using industry-grade NetBeans IDE that can be deployed on GlassFishv2.

Here are is a collection of useful links:

Building Software Factories Today

By Daniel Cazzulino

Software factories is an elusive concept nowadays. In this article, I will try to help you understand which technologies can be used today to build production-quality factories and the reasons for the perceived inconsistency in the tools and the vision on Microsoft's part.

None of what follows is speculation: I've been a main developer on one of the two core technologies that realize today' software factories (patterns & practices Guidance Automation Toolkit or GAT), and as such I've been in close contact with the teams and architects building the technologies I will explain here (including Jack Greenfield, Wojtek Kozaczynski, Mauro Regio and others). Also, my company is a leader in software factory development, providing most of the people that build the factories coming out of the most active practitioner in the area today: the Microsoft patterns & practices group. We also have several customers around the world that we helped reap the amazing productivity boost that today's factory tools can bring.

First and foremost: this article is NOT about the grand vision (still underway, rest assured, but nevertheless in the realm of the future) outlined in The Book: Software Factories: Assembling Applications with Patterns, Models, Frameworks, and Tools by Jack Greenfield, Keith Short et all. I will talk about technologies that are ready for production today. This also means that the strict concept itself of a SF outlined in the book needs to relaxed for what's currently possible with the available tools. You can think of today's factories as infants that will grow with features as times goes by and the end-to-end SF vision becomes a reality. That doesn't prevent the "infant" from giving you a big boost in terms of productivity and software quality.

You can build SF today by using a combination of the following two tools:

  • DSL Tools: or Domain Specific Language Tools. This is a toolkit for creating VS-integrated graphical notations for a given model that you basically make up. It allows you to construct a custom-tailored designer for a model for your problem domain.
  • GAT/GAX: or Guidance Automation Toolkit/Extensions. This is a toolkit for extending VS with solution, project and item templates, code snippets and so-called recipes (or developer use cases) exposed as context menus within VS, that guide the developer in completing a complex task.

The problem is that integrating the two is not a trivial task. Why you may ask? Aren't they both coming from Microsoft? And how does GAT/GAX fit in the SF vision? After all it's not even mentioned in The Book!

Glad you asked ;). In order to understand this "mess", and figure out how to make it work for you, you need to understand a little bit of the background behind the two technologies. The DSL Tools is basically the first step in the road to realizing the full SF vision outlined in The Book. Hence, if you try to build a real-world factory just using DSLs, you will find it fairly incomplete and not supporting typical scenarios such as providing an initial solution structure where you will put your DSLs, etc. The missing features, though, are most probably those that will come (or become unnecessary) as the full vision is implemented. In the meantime, it feels lacking.

On the other hand, Microsoft patterns & practices (p&p) is NOT about building a compelling but somewhat distant feature, but about solving customers' problems today. p&p follows a process called "Customer Connected Engineering", which means customers get to influence what we do. Delivering guidance on how to best use the platform has been p&p traditional focus, but as guidance became more and more complex, thicker books and more complex libraries (think CAB and EntLib) just didn't cut it anymore. Customers were not getting the guidance in an easy to consume way. Hence, we set out to find a way to bring the guidance into Visual Studio, in a way that was easy to author (not only for p&p but for other companies delivering guidance internally) and contextual. The readily available VS extensibility mechanisms (add-ins and VSIP packages) did not satisfy either requirement, so we built our own framework on top of VSIP to allow for easier extensibility. And GAX was born. As the name implies, it's a mechanism for automating the guidance that you would have to otherwise perform manually by following a book's instructions.

You may notice that p&p software factories do not contain DSLs. This is in part because the goals of p&p SF are somewhat different than The Book SF. Their main purpose is to guide the developer through the best practices for implementing a given solution. In the end, it's the developer himself who implements the solution. It just so happens that he does so much faster and in a predictable way because he uses automated guidance within VS that generates code, unfolds boilerplate projects and solutions, provides code snippets, etc.

Given the different origins and goals of the two technologies, it may seem that they are hopelessly irreconcilable. Fortunately, that's not necessarily the case, and there are many ways in which they can be integrated. My partner Victor Garcia Aprea and Mauro Regio have published an article on MSDN about integration scenarios that include integrating the two.

A number of integration points are particularly critical:

  • An initial solution structure is generally required: this can be done with GAX.
  • Typically at a specific location in the solution, a domain model/designer could be added to design some area of the application (i.e. a business entities domain model, that generates business entities + data transfer objects + data access layer code + SQL files with stored procs for the DB). This needs to be done with a combination of DSL and GAX if you want the DSL to only be available at a specific location in the solution. (remember, GAX is about context)
  • With the DSL in place, you may want to generate multiple code files (as in the case of the business entities model I mentioned above), maybe even in different projects, and maybe you even need to generate the projects themselves. Again, DSL tools alone is not enough. By default, it will do two things that are pretty bad for a real-world factory:
    1. The code (a T4 template) that transforms the domain model in a designer into actual code for your project is expected to live in the same project as the domain model file. You are given a custom tool that runs the transformation, and a solution explorer toolbar icon to run all the transformations in the solution/project ("Transform All Templates"). What kind of guidance are you giving your developers if they can modify it at will just by modifying the T4 template that generates their code? It's more like giving them an example that they can tweak, and typically you don't want your developers modifying the templates for code generation that your architects together with your senior devs have created to ensure quality code.
    2. Each T4 template results in exactly one big file with the output of running the codegen against the domain model. If your model designs business entities, you will end up with a huge file with all the entities in the model. Generally not a good idea, and certainly goes against the typical .NET coding practice of using one file per class.

And it will not do the other two things that you need: generate new projects, and generate files that target multiple projects. You can supplement the DSL with GAX in order to achieve all of this. Achieving the above two goals requires deep understanding of how both technologies work at a very granular level, though, but it's definitely possible (more on this later).

  • You may want to launch recipes/execute actions from specific shapes in the domain model diagram (i.e. "Configure Security for Entity" or something like that, maybe even regenerate code for a specific model entity and not all of them). You can hook GAX recipes to the context menus on DSL diagrams pretty easily, and with some more work you can make it contextual enough to provide very granular visibility (i.e. only model entities that have property Foo = true).

Of course, the integration is not as seamless as it could have been. But I hope I shed some light on the current situation by sharing the background and how the two tools came to be. Moving forward, you can expect the experience to become much better, as the scenarios that led to GAT/GAX are integrated into the SF toolkit.

As you might have guessed, we (Clarius) have gone over each and every hop in this integration that you can possibly imagine. We have recently released a CTP of our Software Factories Toolkit, which helps integrate the two technologies, and which you can download for free.


Visual Studio 2005 Service Pack 1 (SP1)

Visual Studio 2005 represents one of the most significant developer tools releases since the introduction of Visual Studio nearly a decade ago. It offers all types of developers a wider spectrum of development possibilities and a greatly advanced design experience. By expanding the Visual Studio product line to include the Visual Studio Team System and the Visual Studio Express Editions, Visual Studio 2005 delivers a compelling toolset for everyone, from hobbyists to enterprise developer teams.

Visual Studio 2005 Service Pack 1 continues Microsoft’s investment in market leading development tools. Service Pack 1 addresses issues that were found through a combination of customers and partner feedback, as well as internal testing. The issues addressed range in severity from places where the syntax coloring was incorrect to customer reported crashes in various scenarios. In some areas, more than 50% of the bugs addressed were reported by customers through the MSDN Product Feedback Center and Microsoft Connect. Overall, Service Pack 1 offers customers improvements in responsiveness, stability and performance for Visual Studio 2005.

Through further advancement and feedback, Service Pack 1 also provides over 70 improvements for common development scenarios including:

  • New processor support (e.g., Core Duo) for code generation and profiling
  • Performance and scale improvements in Team Foundation Server
  • Team Foundation Server integration with Excel 2007 and Project 2007
  • Tool support for occasionally connected devices and SQL Server Compact Edition
  • Additional support for project file based Web applications
  • Windows Embedded 6.0 platform and tools support

For developers using Visual Studio 2005 on Windows Vista, Microsoft is in current development on an update to Service Pack 1 called the ‘Visual Studio 2005 SP1 Vista Refresh Beta’. This update builds on the improvements made in SP1 and delivers a first class experience for developers wanting to take advantages of the new features in Windows Vista. The Visual Studio 2005 SP1 Update for Windows Vista is expected to ship after the consumer availability of Windows Vista in Q1 of 2007 and is now available in beta.

For more information, see the Microsoft Download Center: