B# .NET Blog

Everyone knows UAC by now. One core element of Vista's UAC vision is the elevation of processes, used to run something as an administrator. Typically, you open up a command line instance (cmd.exe) which has been elevated already prior to invoking commands that require elevation. But what if you're already in a non-elevated command prompt window and need to start an executable that runs elevated? In a such a scenario a tool like the one shown below might be useful:

using System;
using System.Diagnostics;
using System.Reflection;

class Program
{
    static void Main(string[] args)
    {
        if (args.Length == 0)
        {
            Console.WriteLine("Usage: {0}.exe <command>", Assembly.GetExecutingAssembly().GetName().Name);
            return;
        }

        string @params = "";
        if (args.Length > 1)
            @params = string.Join(" ", args, 1, args.Length - 1);

        ProcessStartInfo psi = new ProcessStartInfo(args[0], @params);
        psi.Verb = "runas";
        Process.Start(psi);
    }
}

This tool uses the regular Process class from the .NET Framework but specifies a Verb on the ProcessStartInfo object. You can find more info on verbs in the Windows SDK under "Verbs and File Associations". Enjoy!

You've probably already heard about this new feature in .NET 3.5: extension methods. If not, check out a few posts on this topic:

• C# 3.0 Feature Focus - Part 4 - Extension Methods
• Visual Basic 9.0 Feature Focus - Extension Methods

As a quick recap, extension methods allow you to "extend" an existing type with additional methods, without touching the type's implementation itself. In fact, it's nothing more than syntactical sugar on top of static methods. The code below shows such a set of extension methods written in C# 3.0:

using System;

namespace MyExtensions
{
    public static class Extensions
    {
        private static Random rand = new Random();

        public static string Reverse(this string s)
        {
            char[] c = s.ToCharArray();
            Array.Reverse(c);
            return new string(c);
        }

        public static string Permute(this string s, int n)
        {
            char[] c = s.ToCharArray();

            for (int i = 0; i < n; i++)
            {
                int a = rand.Next(s.Length);
                int b = rand.Next(s.Length);
                char t = c[a];
                c[a] = c[b];
                c[b] = t;
            }

            return new string(c);
        }

        public static int Power(this int i, int n)
        {
            int res = 1;
            for (int j = 0; j < n; j++)
                res *= i;
            return res;
        }
    }
}

These methods allow you to write the following, if the MyExtensions namespace is imported:

string name = "Bart";
string reverseName = name.Reverse(); //traB
string fuzzyName = name.Permute(1); //can produce different results where two letters are switched, e.g. Brat
int n = 2;
int kb = n.Power(10); //1024

So far so good. But what does all of this magic have to do with PowerShell? The answer: nothing (so far). As you know, Windows PowerShell 1.0 was developed in the .NET 2.0 timeframe, so it would be very strange if PowerShell understood extension methods. And indeed, it doesn't ... which made me think how we could make this available using some plumbing. Here's what I came up with.

Windows PowerShell has a feature called the Extended Type System. This allows types to be extended with additional properties, methods, etc in order to provide additional IT admin convenience. For example, take a look at the types.ps1xml file in the %windir%\system32\WindowsPowerShell\v1.0 folder on your system. In there you'll find things like:

<Type>
    <Name>System.Array</Name>
    <Members>
        <AliasProperty>
            <Name>Count</Name>
            <ReferencedMemberName>Length</ReferencedMemberName>
        </AliasProperty>
    </Members>
</Type>

This defines an "alias property" which adds a property called Count to each instance of System.Array, pointing to the Length property available on System.Array. So, you can write this:

> $a = "Bart", "John"
> $a.Count
2

In fact, if you use get-member on $a, you'll see the AliasProperty listed out there (click to enlarge):

In a similar way, one can make different types of extensions: Alias Properties, Code Properties, Note Properties, Script Properties, Code Methods, Script Methods. Take a closer look at types.ps1xml for additional samples. Back to our mission now. It seems ETS is an appropriate vehicle to make extension methods available using Script Methods. Basically, we'll provide a script for each extension method that takes the set of original parameters and rewrites these to become parameters of the static method. For example, if you write:

> $name = "Bart"
> $name.Reverse()

the last call should become:

> [MyExtensions.Extensions]::Reverse($name)

Similarly, a Power call on an int should be translated from:

> $n = 2
> $n.Power(10)

into:

> [MyExtensions.Extensions]::Power($n, 10)

Taking possible method overloads into account, we should end up with something like this:

<?xml version="1.0" encoding="utf-16"?>
<Types>
  <Type>
    <Name>System.String</Name>
    <Members>
      <ScriptMethod>
        <Name>Reverse</Name>
        <Script>
          switch ($args.Count) {
            0 { [MyExtensions.Extensions]::Reverse($this) }
            default { throw "No overload for Reverse takes the specified number of parameters." }
          }
        </Script>
      </ScriptMethod>
      <ScriptMethod>
        <Name>Permute</Name>
        <Script>
          switch ($args.Count) {
            1 { [MyExtensions.Extensions]::Permute($this, $args[0]) }
            default { throw "No overload for Permute takes the specified number of parameters." }
          }
        </Script>
      </ScriptMethod>
    </Members>
  </Type>
  <Type>
    <Name>System.Int32</Name>
    <Members>
      <ScriptMethod>
        <Name>Power</Name>
        <Script>
          switch ($args.Count) {
            1 { [MyExtensions.Extensions]::Power($this, $args[0]) }
            default { throw "No overload for Power takes the specified number of parameters." }
          }
        </Script>
      </ScriptMethod>
    </Members>
  </Type>
</Types>

Once we have such a file, it can be "imported" in Windows PowerShell using the Update-TypeData cmdlet:

All we have to do is call this cmdlet as follows:

> Update-TypeData -prependPath MyExtensions.ps1xml

But it gets even better: this chunk of XML is something that's an ideal candidate for dynamic code XML generation. Guess what, let's use LINQ for this task and wrap the "extension method export" functionality in a custom cmdlet:

using System;
using System.IO;
using System.Linq;
using System.Management.Automation;
using System.Reflection;
using System.Runtime.CompilerServices;
using System.Text;
using System.Xml;
using System.Xml.Linq;

namespace BdsSoft.PowerShell.ExtensionMethods
{
    [Cmdlet("Import", "ExtensionMethods")]
    public class ImportExtensionMethods : PSCmdlet
    {
        [Parameter(Mandatory=true)]
        public Assembly Assembly { get; set; }

        [Alias("ns"), Parameter(Mandatory=false)]
        public string Namespace { get; set; }

        protected override void ProcessRecord()
        {
            if (Namespace == null)
                Namespace = "";

            var res =
                new XDocument(
                    new XElement("Types",
                        from t in Assembly.GetTypes()
                        where t.Namespace != null && t.Namespace.StartsWith(Namespace)
                        from m in t.GetMethods(BindingFlags.Public | BindingFlags.Static)
                        where m.GetCustomAttributes(typeof(ExtensionAttribute), false).Length == 1
                        group m by m.GetParameters()[0].ParameterType into g
                        select
                            new XElement("Type",
                                new XElement("Name", g.Key.FullName),
                                new XElement("Members",
                                    from m in g
                                    group m by m.Name into h
                                    select
                                        new XElement("ScriptMethod",
                                           new XElement("Name", h.Key),
                                            new XElement("Script", GetScriptFor(h))
                                    )
                                )
                            )
                    )
                );

            StringBuilder sb = new StringBuilder();

            using (TextWriter tw = new StringWriter(sb))
            {
                using (XmlTextWriter xtw = new XmlTextWriter(tw))
                {
                    xtw.Indentation = INDENT;
                    xtw.Formatting = Formatting.Indented;
                    res.WriteTo(xtw);
                }
            }

            base.WriteObject(sb.ToString());
        }

        static int INDENT = 2;

        static string GetScriptFor(IGrouping<string, MethodInfo> m)
        {
            StringBuilder sb = new StringBuilder();
            sb.AppendFormat("\r\n{0}switch ($args.Count) {{\r\n", new String(' ', INDENT * 5));

            foreach (var e in m)
            {
                int n = e.GetParameters().Length - 1;
                sb.AppendFormat("{0}{1} {{ {2} }}\r\n", new String(' ', INDENT * 6), n, GetScriptFor(e.DeclaringType.FullName, e.Name, n));
            }

            sb.AppendFormat("{0}default {{ throw \"No overload for {1} takes the specified number of parameters.\" }}\r\n", new String(' ', INDENT * 6), m.Key);
            sb.AppendFormat("{0}}}\r\n{1}", new String(' ', INDENT * 5), new String(' ', INDENT * 4));

            return sb.ToString();
        }

        static string GetScriptFor(string type, string method, int n)
        {
            StringBuilder sb = new StringBuilder();
            sb.Append("$this");

            for (int i = 0; i < n; i++)
                sb.AppendFormat(", $args[{0}]", i);

            string args = sb.ToString();

            return String.Format("[{0}]::{1}({2})", type, method, args);
        }
    }
}

Quite a bit of code, but lots of plumbing to get a smooth output (notice the code can be improved in many places). The GetScriptFor methods are pretty simple to understand and generate the script for a given (group of) method (overloads) associated with a method name (the second GetScriptFor method is a helper to get the method calls themselves, using the $this and $args variables). For what the core functionality is concerned, take a look at the ProcessRecord method that contains a LINQ query that looks for all extension methods in the given assembly and namespace:

            var res =
                new XDocument(
                    new XElement("Types",
                        from t in Assembly.GetTypes()
                        where t.Namespace IsNot Nothing AndAlso t.Namespace.StartsWith(Namespace)
                        from m in t.GetMethods(BindingFlags.Public | BindingFlags.Static)
                        where m.GetCustomAttributes(typeof(ExtensionAttribute), false).Length == 1
                        group m by m.GetParameters()[0].ParameterType into g
                        where !g.Key.IsGenericType
                        select
                            new XElement("Type",
                                new XElement("Name", g.Key.FullName),
                                new XElement("Members",
                                    from m in g
                                    group m by m.Name into h
                                    select
                                        new XElement("ScriptMethod",
                                           new XElement("Name", h.Key),
                                            new XElement("Script", GetScriptFor(h))
                                    )
                                )
                            )
                    )
                );

In here, we're using the new System.Xml.Linq API to contruct an XML fragment on the fly (LINQ to XML style). A method is considered to be an extension method if it's public, static and marked with a System.Runtime.CompilerServices.ExtensionAttribute custom attribute. Furthermore, it's first parameter (the "this" parameter in C# 3.0) shouldn't be a generic type, since PowerShell doesn't have first-level support for generics at the moment (this restriction rules out the use of System.Core's extension methods unfortunately, at the moment). Using the grouping constructs, methods are grouped per type and per method name (to account for overloads). VB folks have an easier job when it comes down to generating XML fragments. The query above looks as follows in VB 9.0 (more info on VB 9.0 XML integration):

When writing a cmdlet, we need a snap-in as its distribution vehicle; below is a simple one:

using System.ComponentModel;
using System.Management.Automation;

namespace BdsSoft.PowerShell.ExtensionMethods
{
    [RunInstaller(true)]
    public class ExtensionMethodsSnapIn : PSSnapIn
    {
        public override string Description
        {
            get { return "This Windows PowerShell snap-in provides support for .NET Framework 3.5 Extension Methods."; }
        }

        public override string Name
        {
            get { return "BdsSoft.PowerShell.ExtensionMethods"; }
        }

        public override string Vendor
        {
            get { return "BdsSoft"; }
        }
    }
}

Strong-name the assembly, build it and run installutil -i against the generated dll file. Finally, create the following PowerShell script, ImportExtensionMethods.ps1:

if ($args.Count -lt 2)
{
   throw "Usage: .\ImportExtensionMethods.ps1 output assembly [namespace]"
}
else
{
   $emOutput = (join-path (split-path $profile) $args[0])

   Import-ExtensionMethods -assembly $args[1] -namespace $args[2] | Out-File $emOutput
   Update-TypeData -PrependPath $emOutput
}

This simplifies calling the Import-ExtensionMethods cmdlet and to update the ETS type data, so that the extensions become available. In the screenshot below, you can see the whole thing in use:

All you need to do is:

1. Make sure the snap-in is loaded, using add-pssnapin (you can move this call to your PowerShell profile if you want to load it automatically).
2. Call ImportExtensionMethods.ps1, passing in a name for the ETS ps1xml file that should be generated followed by the assembly that contains the extension methods (use Assembly.LoadFrom or Assembly.Load to get the assembly either by file name or by assembly name, i.e. for GAC'ed assemblies).

Of course, once you have the ps1xml files, you could simply adapt your PowerShell profile file in order to load the assemblies and call Update-TypeData to load the ps1xml file. In the next screenshot you can see the generated ps1xml file (click to enlarge):

Have fun!

Welcome back to the Visual Basic 9.0 Feature Focus blog series. In this post, we'll (finally) cover one of .NET Framework 3.5's core features for both C# 3.0 and VB 9.0: LINQ. LINQ stands for Language INtegrated Query and provides querying syntax directly in a general-purpose programming language. Combined with clean and elegant APIs like LINQ to Objects, LINQ to SQL and LINQ to XML (amongst others, like LINQ to SharePoint) this gives developers all the horsepower they need to retrieve and manipulate data from any LINQ-capable data source, also closing the gap between various data models and objects from the world of OO (for instance, by having entity mappings for relational SQL databases, a.k.a. O/R mapping).

So, what's the big idea? In the past, there were various problems related to working with data. One of those problems is the programming language's inawareness of the concept of queries. For example, you typically write code like this:

Using conn As New SqlConnection(dsn)
    Using cmd As New SqlCommand("SELECT * FROM Products WHERE UnitPrice > 100 ORDERBY UnitsInStock DESCENDING", conn)
        Using reader = cmd.ExecuteReader()
            While reader.Read()
                Dim name As String = CType(reader("ProductName"), String)
                Dim price As Decimal = CType(reader("UnitPrice"), Decimal)
                'Use it
            End While
        End Using
    End Using
End Using

What's wrong with this? A couple of things in fact. First of all, the SQL statement is wrapped inside a string which makes it impossible for the compiler to analyze it and check it against the database schema for correctness (well, in theory we could have additional compiler intelligence, like the printf format string checking in many C/C++ compilers, but that doesn't solve the fundamental problem). Secondly, inside the iteration statement, we're faced with weak typing problems: ProductName was a string, is masked as an object and has to be casted back to a String in order to work with it (which doesn't seem too much of a problem if you're working with late binding semantics, but it would be better if we could have strong typing bubbling up to the surface). Another typical problem is query statement parameterization which can be done but (in lots of case) wrong as well. What I'm referring at is a phenomenon known as SQL Injection Attacks that occurs if you're building query statements by simple string concatenation, instead of using parameterized statements or sprocs. Last but not least there's all of this awful plumbing around the query statement in order to execute it and to iterate over the results, which by the way aren't real results: it's a bit of loosely coupled data that you still have to tie together using some O/R mapping to live in an ideal world. So, how can this be improved?

The approach taken by LINQ is unique in many fashions, the core asset being the LIN part: Language INtegrated. Furthermore, with LINQ we do have one single syntax for all sorts of queries, including in-memory ones and queries targeting various other kinds of data sources, like SQL, XML, web services like Amazon, SharePoint, ... In this post, we'll take a closer look at the query syntax and what it corresponds to in terms of extension method calls. Be prepared to get a big deja-vu of most features we did discuss in previous posts, including Extension Methods, Lambda Expressions, Expression Trees, Anonymous Types, Object Initializers, etc. Let's start with a very simple query using LINQ to Objects, otherwise known as in-memory queries:

Dim ints As Integer() = {1, 2, 3, 4, 5}
Dim res = From i In ints Where i Mod 2 = 0 Select i
For Each i In res
    Console.WriteLine(i)
Next

Note: A common question I often get is why the query starts with a From clause, in contrast to SQL statements. The reason why isn't difficult at all: by having the From clause first, the IDE knows about the source you're about to query. Since the source should be an IEnumerable(Of T), each time you refer to an object from that source using the dummy name from the From clause (the .. part in From .. In ...), it knows that one is of type T (from IEnumerable(Of T)). So, the editor can provide IntelliSense on the query statements. If you were to start with a Select clause, how can the editor know what the members are of i (Select i. --> can you suggest any meaningful IntelliSense, ignoring the System.Object members of course?).

The query will return all even numbers, but how does it really work? Let's do what the compiler does: translate the query to another format:

Dim res = ints.Where(Function(i) i Mod 2 = 0).Select(Function(i) i)

Essentially this is a chain of extension method calls on the IEnumerable(Of T) object, combined with lambda expressions. In compiling the original query, the compiler looks for the dummy variable in the From .. In clause and uses that one as the parameter to all subsequent lambdas. Next it takes the expressions followed by Where .., Select .., etc and cooks a lambda out of it, by taking the dummy as the parameter and the original expression as the lambda body. Finally it chains the whole thing together using extension methods calls. Now on to these extensions: what does Where and Select do? First of all, both are extensions to IEnumerable(Of T) and have been defined in System.Linq.Enumerable. So, the code above is equal to:

Dim res = Enumerable.Select(Enumerable.Where(ints, Function(i) i Mod 2 = 0), Function(i) i)

Note: you might wonder why there's a Select call with a trivial projection lambda Function(i) i. If the original item is returned, what's the value of this? The answer is to make a query really a query, i.e. to 'defeat' degenerate queries. If a trivial query like "From i In ints Select i" would be optimized to just "ints" (because the projection doesn't do anything), the source of the query is revealed to the consumer of the query, something which has to be avoided in order to have a clean design (and which is more important when talking about LINQ to SQL kind of queries too).

Quite funny isn't it, to see the order of the methods calls being reversed due to the use of extension methods (tip: people who saw a decent amount of lambda calculus and formal semantics can write down some nice formulas to generalize this idea). The signature of these extension methods is the following:

Function Where(Of T)(ByVal source As IEnumerable(Of T), ByVal predicate As Func(Of T, Boolean)) As IEnumerable(Of T)
Function Select(Of T,R)(ByVal source As IEnumerable(Of T), ByVal projection As Func(Of T, R)) As IEnumerable(Of R)

Both generic methods take in a source as well as a function delegate that transforms the input to something else: the predicate returns a Boolean (should the item be included or not?) and the projection returns an object of another type R (the result of projection T into something else, e.g. an anonymous type).

Evaluation starts with the Enumerable.Where call, which takes the source from its first parameter and iterates over it, evaluating the lambda expression "predicate" for each and every item in the source sequence. If that predicate evaluates to true, it returns the item in the result sequence, which also is of type IEnumerable(Of T). Unfortunately I can't give you sample code of how the Where method works since Visual Basic doesn't have a concept called "iterators" which lies at the basis of this mechanism and is a C# 2.0 (and higher) feature. What is enables is a thing called "lazy evaluation": only when you start iterating over the results of the method, data is fetched from the source. It's like having a pipeline that sucks data out of the source only when it's really needed:

In the picture above imaging you start iterating over the results of the Select (the end of the query in the sample). As you turn the Select wheel, the Where weel will start to spin around, getting data from the source sequence till a match is found. Then that match bubbles up all the way to the Select output. So, in the source sequence, the following will happen:

1, 2, 3, 4, 5

You start to iterate over Select, which comes down to calling MoveNext on the enumerator (the thing that drives the For Each loop). Which enumerator? The one returned by the Select call. Internally, the Select method sucks data from the Where method by calling the Where method's returned enumerator's MoveNext method. This on its turn triggers an iteration over the source sequence. We've just arrived at the bottom of the call stack, now let's make our way up again:

• 1 is fetched from the source
  • Where evaluates the predicate and sees 1 doesn't match
• 2 is fetched from the source
  • Where evaluates the predicate and sees 2 is a match, so it yields back the 2 to its caller
    • Select got 2 from its source, it executes the lambda on it and returns the result (the lambda "projection" being Function(i) i, thus returning i itself)
      • The caller of the iteration receives 2

It's until the caller asks for another iteration cylce that the whole thing is suspended. That's why we call this lazy loading. Assume you have a source sequence of 1 million items and you break the iteration after five cycles: you won't have loaded any more data from the source sequence than is required. This is different than doing simple collection manipulations, reading the entire source sequence and finding all elements that are even, then taking this result and doing the projection for each of those items. In such a case you'd end up with a maximum of 1 million items (all the odd numbers are filtered out), maybe 500,000 or so, but you just needed 5 in the final iteration. So you've just spilled lots of processor cycles and memory space to do more than you really needed to

Note: There are situations however where lazy loading is broken because all elements are needed to carry out some operations. Notable examples are sorting and grouping. Indeed, to sort a sequence of items that have been ordered at random, all items have to be fetched in order to carry out a sort, before the result sequence can be returned to the caller.

So far so good for in-memory queries. What about other queries, like LINQ to SQL queries? Again, the mechanism is very similar. However, a query like the following:

Dim ctx As New NorthwindDataContext()
Dim res = From p In ctx.Products Where p.UnitPrice > 100 Select p.ProductName
For Each p In res
    Console.WriteLine(p)
Next

The first stage of the compilation is the same as previously:

Dim res = ctx.Products.Where(Function(p) p.UnitPrice > 100).Select(Function(p) p.ProductName)

Now it becomes interesting. As you know, Where and Select are extension methods but how does the compiler find the appropriate one? The answer is simple: by looking at the variable on which the method is called. In this case ctx.Products is of type Table(Of T), a class defined in the System.Data.Linq namespace. Table(Of T) doesn't have an instance method called Where, so the compiler starts to look for extension methods, a search based on the type of the object. There are no extension methods for Table(Of T) in scope but since Table(Of T) implements IQueryable(Of T), a class from the System.Linq namespace, the compiler looks for extension methods on IQueryable(Of T) and finds that one is available now (hooray). The signature of this method (and the Select method, since Queryable.Where returns an IQueryable(Of T) again) is:

Function Where(Of T)(ByVal source As IQueryable(Of T), ByVal predicate As Expression(Of Func(Of T, Boolean))) As IQueryable(Of T)
Function Select(Of T,R)(ByVal source As IQueryable(Of T), ByVal projection As Expression(Of Func(Of T, R))) As IQueryable(Of R)

So, the query is now translated into:

Dim res = Queryable.Select(Queryable.Where(ctx.Products, Function(p) p.UnitPrice > 100), Function(p) p.ProductName)

If you read the method signatures carefully, you'll see that the type of res is now IQueryable(Of String), because p.ProductName (the result value of the method call chain produced by the original query) is a string. However, there's more. Notice the second parameters of both methods: these are no longer of type Func(Of ...) but of type Expression(Of Func(Of ...)), which - as we learned before in the episode on Expression Trees - trigger the compiler to generate expression trees for the lambdas. So,

Function(p) p.UnitPrice > 100
Function(p) p.ProductName

both become expression trees, passed in to the IQueryable(Of T) object. Now, when you trigger iteration over the query (results), like this:

For Each p In res
    Console.WriteLine(p)
Next

or indirectly, e.g. by doing databinding sort of operations, you'll cause the IQueryable(Of T) implementation to parse the expression trees and transform these into the appropriate target language, in this case SQL:

SELECT p.ProductName FROM Products WHERE p.UnitPrice > 100

Now you know the basic principles, it's time to take a look at the various query operators that are supported by LINQ (yes, there's more than just Wheres and Selects):

From indicates the source of a query and has the following syntax:

From <dummy> [As <type>] In <source>

• dummy is a variable name for use throughout the query to denote an item being processes (e.g. in a predicate or in a projection);
• type can be used to indicate the type of the dummy, i.e. the type of an item in the source sequence; it's not mandatory if the type of the item can be inferred (e.g. when using generic collections IEnumerable(Of T) or IQueryable(Of T));
• source obviously references the query source, e.g. an in-memory collection or an IQueryable(Of T) object.

Note: More than one From clause can be used in a query.

Select defines a projection clause in a query:

Select <projection>

• projection can be any expression that returns one object; it could be a dummy from a From clause or something more complex, using the syntax below:

[<alias> = ] <column>[,...]

• alias is optional and indicates the name of the property that will be generated on the anonymous type representing the projection;
• column refers to a property or field on a dummy but could be a more complex expression too (in which case an alias is required since no name can be inferred automatically).

Examples:

• Dim res = From i In New Integer() {1, 2, 3, 4, 5, 6} Select i 'returns 1,2,3,4,5,6
• Dim res = From i In New Integer() {1, 2} Select Double = 2 * i, Triple = 3 * i 'returns {Double = 2, Triple = 3}, {Double = 4, Triple = 6}

Where is used to filter results based on a predicate:

Where <predicate>

• predicate is a Boolean-valued expression that can use every variable in scope, including the dummies defined in From clauses.

Example: Dim res = From i In New Integer() {1, 2, 3, 4, 5, 6} Where i Mod 2 = 0 Select i 'returns 2,4,6

Order By clauses put an ordering on the query's results:

Order By <ordering> [Ascending | Descending]

• ordering is an expression used a the key in an ordering; if multiple orderings are specified in a row, subsequent orderings define secondary, ternary, ... and n-ary orderings (for the curious ones, primary orderings are translated into OrderBy and OrderByDescending calls, while n-ary orderings are translated into ThenBy and ThenByDescending calls).

Example:

• Consider src contains { ID = 1, City = "Ghent", Country = "Belgium" }, { ID = 2, City = "Brussels", Country = "Belgium" }, { ID = 3, City = "Seattle", Country = "USA" }
• Dim res = From c In src Order By c.Country Descending Order By c.City Select c.ID 'returns 3,2,1

Distinct filters out duplicates from a source sequence in a query:

Distinct

• returns all distinct items from the input of the clause (i.e. "everything in front of" the Distinct clause)

Example: Dim res = From i In New Integer() {1, 2, 1, 3} Select i Distinct 'returns 1,2,3

Skip and Take allow to select a range of items from a source sequence in a query:

Take(<number>)
Skip(<number>)

• number is an integer value; e.g. Skip(10).Take(5) will select items 11 to 15 from the input sequence

Example: Dim res = From i In New Integer() {1, 2, 3, 4, 5, 6} Select i Skip 3 Take 2 'returns 4,5

Aggregate performs an aggregate, i.e. turning a sequence of items into a singleton value:

Aggregate <dummy> In <source> Into <aggregationList>

• dummy is a dummy name for use in the aggregation;
• source can be anything which is enumerable (including a nested subquery);
• aggregationList is a comma-separated set of aggregation expressions that take the form:
  
  [<alias> =] <aggregation>[,...]
  
  If more than one aggregation expression is present, an anonymous type will be generated using the aliases used in the aggregation expression list. The aggregation supports various functions including Min, Max, Sum, Average, ...

Example: Dim res = Aggregate i In (From i In New Integer() {1, 2, 3, 4, 5, 6} Where i Mod 2 = 0 Select i) Into a = Average(), s = Sum() 'returns res.a = 4, res.s = 12

Group is used to make groupings based on a grouping expression; this is slightly more complicated than the query operations above:

Group [<fields>] By <keys> Into <aggregates>

• fields is an optional comma-separated list and allows to specify the fields that have to be included in the grouped results (you could consider this to be a pre-grouping Select-alike projection);
• keys is a comma-separated list of one or more name = value pairs, specifying the grouping keys;
• aggregates is a comma-separated list of expressions that make up the group result and allow for continuation of the query.

A sample will clarify things so much:

Dim res = From s In New String() {"Bart", "Bob", "John"} Group By FirstLetter = s(0) Into Words = Group
For Each letter In res
    Console.WriteLine(letter.FirstLetter)
    For Each word In letter.Words
        Console.WriteLine("- " + word)
    Next
Next

Notice we don't have a fields part in here, but in most cases queries as the one above are complex enough for day-to-day purposes. In this sample, a set of words are grouped by the first letter (FirstLetter) of the word and the items in the group are called Words. Inside the loop, we can refer to FirstLetter as the grouping key and Words as the collection of the items in the corresponding group.

Join allows to join multiple sources together based on some join condition:

Join <dummy> In <source> On <key1> Equals <key2> [And ...]

• dummy refers to items in the to-be-joined source;
• source is the source of the items that have to be joined;
• key1 and key2 are expressions to used in the join condition.

An example based on Northwind looks like this:

Dim res = From p In ctx.Products Join s In ctx.Suppliers On p.Supplier = s Select p.ProductName, s.SupplierName

Joins can be more complex if using "group joins" or when multiple join conditions are involved but let's keep it simple for now :-).

Happy coding!

Welcome back to the Visual Basic 9.0 Feature Focus blog series. In this post, we'll cover a feature unique to Visual Basic: XML Support. In VB 9.0, XML has become a first class citizen of the language, meaning you can use XML literals to build XML fragments (sounds pretty logical isn't it?) together with a few operators that make working with XML really simple (and I mean really simple). Combine this with support for XSD schemas and you get a superb language feature. Let's take a closer look...

First of all, take a look at the Inline XML syntax:

Dim xml = <Books>
                      <Book Title="Introducing VB 9.0" Price="0.00">
                          <Author>Bart De Smet</Author>
                      </Book>
                      <Book Title="Introducing C# 3.0" Price="0.00">
                          <Author>Bart Simpson</Author>
                      </Book>
                  </Books>

Not very exciting is it? Just plain XML inside the program, combined with and Implicitly Type Local Variable. So, what's the type of the xml variable in the fragment above? Drumroll ... it's an XElement. No, not an XmlElement, but an XElement without the 'ml'. XElement is defined in a new .NET Framework 3.5 namespace, called System.Xml.Linq that's included by default in new VB 9.0 applications. So, why do we have a new XML API in .NET? The answer is simple: make things easier and composable. If there's one thing the System.Xml implementation - based on the DOM - lacks, it's composability. Imagine you have to construct the XML piece from the code above using the System.Xml API. You'd end up with spaghetti like this:

Dim doc As New XmlDocument()
Dim books As XmlElement = doc.CreateElement("Books")

Dim book1 As XmlElement = doc.CreateElement("Book")
Dim title1 As XmlAttribute = doc.CreateAttribute("Title")
title1.Value = "Introducing VB 9.0"
book1.Attributes.Append(title1)
Dim title2 As XmlAttribute = doc.CreateAttribute("Price")
title2.Value = "0.00"
book1.Attributes.Append(title2)
Dim author1 As XmlElement = doc.CreateElement("Author")
author1.InnerText = "Bart Simpson"
book1.AppendChild(author1)
books.AppendChild(book1)

The code fragment above just defines one single book by the way... I do have only one question for you: can you still smell the XML in the code above? Apart from the language integration in VB 9.0, the new System.Xml.Linq API is much more composable. To illustrate this, consider the fragment below:

Dim xml = New XElement("Books", _
            New XElement("Book", _
                New XAttribute("Title", "Introducing VB 9.0"), _
                New XAttribute("Price", "0.00"), _
                New XElement("Author", "Bart De Smet")), _
            New XElement("Book", _
                New XAttribute("Title", "Introducing C# 3.0"), _

                New XAttribute("Price", "0.00"), _
                New XElement("Author", "Bart Simpson")))

This piece of code defines the same as the original fragment that uses VB 9.0's Inline XML feature. I'm sure you agree this, compared to the System.Xml code, exposes the XML structure quite fairly well. In fact, the code shown above just has one constructor call at the top level, constructing the "Books" element and specifying the child items. Even more, guess what a call to ToString() returns? Yes, indeed:

That's about it for the System.Xml.Linq API right now, feel free to play around with it and let IntelliSense be your guide. You'll find it very easy to create new pieces of XML even using the raw APIs (that is, without Inline XML in case you want to do the work yourself) and to get to the data. But how can we make this even easier by means of language integration? Let's explore. What about this:

Dim xml = <Books>
                      <Book Title="Introducing VB 9.0" Price="0.00">
                          <Author>Bart De Smet</Author>
                      </Book>
                      <Book Title="Introducing C# 3.0" Price="0.00">
                          <Author>Bart Simpson</Author>
                      </Book>
                  </Books>

For Each book In xml.<Book>
    Console.WriteLine("{0} costs {1} and is written by {2}", book.@Title, book.@Price, book.<Author>.Value)
Next

Whoosh, can it be any easier than this? Just iterate over every <Book> in the XML fragment and select some values. That's right: you can use the dot '.' syntax (the child axis) to dive in the XML structure (a <Book> has an <Author>, thus you can write book.<Author>) and the '.@' syntax (the attribute axis) to select attributes. So, the code above prints:

Introducing VB 9.0 costs 0.00 and is written by Bart De Smet
Introducing C# 3.0 costs 0.00 and is written by Bart Simpson

However, there's something that can be improved in the code above: writing it. Take a look at the IntelliSense:

Although a few syntax elements are outlined, there's not much help from the IDE at this stage of the game. That's because the IDE doesn't have information about the XML schema, so how to solve this? The answer, again, is pretty simple: import an XML schema, just like you import a CLR namespace. But before we can do se, we need to create the schema first. An easy way to do this, is by copying the XML fragment straight from the code to an XML file in Visual Studio. Then go to XML, Create Schema:

Next, copy the generated schema to the Clipboard, add new item of type XML Schema to the solution, named "VB9Demo.xsd":

Copy in the XSD schema that's still on the clipboard and make sure to add the targetNamespace attribute:

Save the schema file and go back to the code file and add the following import on top of the code:

Imports <xmlns="http://tempuri.org/VB9Demo">

As you'll see while typing, IntelliSense pops up and helps you a bit:

 Now, when you start to use the xml variable, IntelliSense will be so kind to help you a bit:

and

Notice it's also possible to denote an XML namespace by using the following:

Imports <xmlns:ns="http://tempuri.org/VB9Demo">

See the :ns part in the Imports statement above? Now you can use <Books>, <Book> and <Author> if you prefix these with ns:, just as in plain XML. If you don't use a prefix, the schema will be considered to be the default one. A final note on IntelliSense for XML: in VB 9.0 this is limited to XML variables that have been constructed already and that have a known namespace in place. For example, when writing a piece of Inline XML, you won't get IntelliSense. This is something the team is looking into for a subsequent release but I think all of this magic is already very very powerful (and you might want to type the XML pieces in a regular XML editor - like Visual Studio itself - that's aware of the schema you're using, so you get IntelliSense when writing the XML; finally you can copy the result right into your code).

There's one operator I didn't mention yet, that's the ... operator (the descendant axis):

For Each author In xml...<Author>
    Console.WriteLine(author.Value)
Next

This will look for all <Author> tags no matter how deep these are nested in the XML structure, which is certainly easier than having to traverse the XML tree manually or using the '.' syntax.

If you think all of this is already very cool on its own, wait for the next post in this series where we'll cover LINQ, including the combination with XML features in VB 9.0 (a.k.a. the return of the <% ... %>).

Happy coding!

Welcome back to the Visual Basic 9.0 Feature Focus blog series. In this post, we'll cover Runtime Agility. Before we continue, I have to warn you: this is a HUGE feature even though it hasn't been covered that much yet in the blogosphere. Historically, Visual Basic has been a language that's highly dependent on a runtime library. People from the old VB days will certainly remember msvbvmxx.dll, the Microsoft Visual Basic Virtual Machine version xx. Not only did this library contain runtime functionality (e.g. for memory management), it also did contain helper functions that are used to gi