Friday, June 8, 2007

Mandelbrot Set Fractal in Fortress

One of my favorite embarrassingly parallel problems is the rendering of the Mandelbrot set fractal. It is a simple program that produces very interesting images.

There are many ways to optimize this program, however I'm going to avoid these optimizations and try to create a naive implementation of the problem, just to see to the program looks like in Fortress. Also I'm going to try to take advantage of the parallel for loops that Fortress provides.


The first thing to define is something to represent complex numbers (remember this is a non-optimize version of the problem) . It seems that in the current reference implementation of there's no native support complex number (that I could find). So this gave me the opportunity to implement it.


value object Complex( real:RR64, img:RR64)
opr+(self,other:Complex) = Complex(real+other.real,img+other.img)
opr juxtaposition(self,other:Complex) =
Complex((real other.real) - (img other.img),
(real other.img) + (other.real img))
toString() = ("Complex(" real ", " img ")")
end

opr |x:Complex| = SQRT(x.real^2 + x.img^2)


complexExp (o : Complex,n : Integral) = do
if (n = 1)
then o
else (o complexExp(o,n - 1))
end
end

opr^(o:Complex,n:Integral) =
if (n = 2)
then (o o)
else complexExp(o,n)
end


Note that this implementation is not complete, only the required elements for the Mandelbrot set are implemented. It is interesting that multiplication in Fortress doesn't use the '*' operation but it uses juxtaposition of elements to be multiplied. That is, instead of saying (x*y) you say (x y).

Then we need something that helps us to convert from screen coordinates to real coordinates. To do this I created the following function:


lFunc(x1 : RR64,y1 : RR64, x2 : RR64, y2 :RR64) = do
m = (y2 - y1) / (x2 - x1)
b = y1 - (m x1)
fn(x) => (m x) + b
end


Note that this function returns another function that preforms the conversion.

Now the following code shows the implementation of the Mandelbrot set algorithm for one row in the image.


lineMandelbrot[\W\](startIndex : ZZ32, endIndex : ZZ32,
lineData :Array[\ZZ32,ZZ32\],
f :RR64 -> RR64, y : RR64) = do

for i <- startIndex:endIndex do
p0 = Complex( f(i) , y)
p : Complex := p0
j : ZZ32 := 0
maxIteration = 255
while ( |p| < 2.0 AND j < maxIteration ) do
p := p^2 + p0
j := j+1
end
lineData[i] := j
end
end


Note that I use a parallel for loop for the external for statement in order to say that each point can be calculated independently in a separate thread.

This code looks very nice when formated with Fortify:



Finally the following code show the main program. For the graphic generation, a PPM file was used since is the easiest image format to generate!.


run(a:String...) =
do
imageWidth = 100
imageHeight = 100
startY = -1.0
endY = 1.0
startX = -1.0
endX = 1.0

image = array[\ZZ32\](imageWidth)

fout: BufferedWriter = outFileOpen "output.ppm"
outFileWrite(fout,"P3\n")
outFileWrite(fout,"" imageWidth " " imageHeight "\n")
outFileWrite(fout,"255\n")


startIY = 1
endIY = imageHeight

fY = lFunc(startIY,startY,endIY,endY)
fX = lFunc(0,startX,imageWidth - 1,endX)

for iY <- seq(startIY#endIY) do
print "Line: " iY "\n"
lineMandelbrot(0,imageWidth - 1,image,fX,fY(iY))

for i <- seq(0#imageWidth) do
outFileWrite (fout,"0 0 " image[i] " ")
end
outFileWrite(fout,"\n")
end
outFileClose(fout)
()
end




Since the reference implementation focus is not performance, I think is not fair to publish benchmarks or something like that. However I noticed that by playing around with the NumFortressThreads environment variable (found by looking at the source code) I got different execution times in my dual core machine .

Sunday, June 3, 2007

Parallel For Loops in Fortress

On interesting and promising feature of Fortress is that for loops are parallel by default. This is documented in the 2.8 For Loops Are Parallel by Default section of the Fortress Language Specification document.

It seems that the reference implementation already supports this feature. For example when running this code:


component ForLoops

export Executable

run(args:String...):() = do
for i <- 1:10 do
print("Iteration " i ""\n")
end
end

end


The output shows:


Parsing /home/ldfallas/blog/fortress/for/forloopexperiment.fss with the Rats! parser: 283 milliseconds
Read /home/ldfallas/fortress/FortressLibrary.tfs: 566 milliseconds
Iteration 10
Iteration 9
Iteration 6
Iteration 4
Iteration 8
Iteration 7
Iteration 2
Iteration 5
Iteration 1
Iteration 3
finish runProgram
1711 milliseconds



According to the specification the generator section of the for loop controls this behavior (section 13.15 and 13.14) . If the sequencial generator is used, the loop is executed in the classic order. For example:


component ForLoops

export Executable

run(args:String...):() = do
for i <- sequential(1:10) do
print("Iteration " i ""\n")
end
end

end


The output is:


Parsing /home/ldfallas/blog/fortress/for/forloopexperimentSeq.fss with the Rats! parser: 282 milliseconds
Read /home/ldfallas/fortress/FortressLibrary.tfs: 546 milliseconds
Iteration 1
Iteration 2
Iteration 3
Iteration 4
Iteration 5
Iteration 6
Iteration 7
Iteration 8
Iteration 9
Iteration 10
finish runProgram
1523 milliseconds

Wednesday, May 30, 2007

Using Scala Extractors with the XSD Schema Infoset Model

In this post I'm going to use Scala extractor objects with the Eclipse XML Schema Infoset Model to identify common ways of defining XML Schemas.

Hopefully I'm going to show that complex patterns on common Java objects can be identified using Scala extractors.

The Eclipse Schema Infoset Model is a complex EMF model the represents the W3C XML Schema. The Analyzing XML schemas with the Schema Infoset Model and Analyze Schemas with the XML Schema Infoset Model articles provide a nice explanation on how to work with this model.

For this post I wanted to create Scala patterns that identify common design patterns in Xml Schemas. There are four common patterns for XML Schemas: Russian Doll, Salami Slice, Venetian Blind and Garden of Eden.

The article Introducing Design Patterns in XML Schemas provide a nice explanation on each of this patterns. Also the article talks about a nice feature of NetBeans Enterprise Pack that allows the user to move a schema from one design pattern to another. More information on these design patterns can be found on the article Global vs Local from the xFront site.

The W3C XML Schema model is huge, but for this post I'm going to consider only a small subset.

The first step is the definition of the extractor objects that will be used to have access to certain properties of the Xml Schema Infoset model.


package langexplr.scalaextractorexperiments;

import org.eclipse.emf.ecore.resource._
import org.eclipse.emf.ecore.resource.impl._
import org.eclipse.xsd._
import org.eclipse.xsd.impl._
import org.eclipse.xsd.util._
import org.eclipse.emf.common.util.URI

object XSDSchemaParts {
def unapply(schema : XSDSchema) =
Some ((schema.getTargetNamespace(),
List.fromIterator(
new JavaIteratorWrapper[XSDTypeDefinition](
schema.getTypeDefinitions().iterator())),
List.fromIterator(
new JavaIteratorWrapper[XSDElementDeclaration](
schema.getElementDeclarations().iterator()))))

}

object XSDElementParts {
def unapply(elementDeclaration : XSDElementDeclaration) =
Some((elementDeclaration.getName(),elementDeclaration.getTypeDefinition()))


}

object XSDComplexType {
def unapply(typeDefinition : XSDTypeDefinition) =
if (typeDefinition.isInstanceOf[XSDComplexTypeDefinition]) {
val complexType = typeDefinition.asInstanceOf[XSDComplexTypeDefinition];
Some((complexType.getName(),complexType.getContent()))
} else {
None
}

}

object XSDSimpleType {
def unapply(typeDefinition : XSDTypeDefinition) = {
if (typeDefinition.isInstanceOf[XSDSimpleTypeDefinition]) {
Some(typeDefinition.asInstanceOf[XSDSimpleTypeDefinition])
} else {
None
}
}
}

object XSDParticleContent {
def unapply(p : XSDParticle) = Some(p.getContent())
}

object XSDSimpleSequenceModelGroup {
def unapply(complexTypeContent : XSDComplexTypeContent) = {
complexTypeContent match {
case XSDParticleContent(mg : XSDModelGroup)
if (mg.getCompositor().getName == "sequence") =>
Some(
List.fromIterator(
new JavaIteratorWrapper[XSDParticle](
mg.getContents.iterator())))
case _ => None
}
}



The XSDSchemaParts, XSDElementParts, XSDComplexType, XSDSimpleType, and XSDParticleContent extractor objects provide access to some properties of a model object. For example the XSDSchemaParts returns a tuple with the target namespace, the complex type definitions and the element definitions.

Also the XSDSimpleSequenceModelGroup provide an easy way to identify a common pattern that is the use of a XSD sequence as the type main element.

A class will be created for each design pattern. The following trait is the base for all of them:


trait XsdDesignPattern {
def name : String
def identify(schema:XSDSchema) : boolean
}



Now we can define each pattern:

Russian Doll

This design pattern says that the structure of the XML Schema is similar to the document structure. Only one public element is defined and all other elements are defined inside of it.

For example:


<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
targetNamespace="http://langexplr.blogspot.com/DocsRussianDoll"
xmlns:p="http://langexplr.blogspot.com/DocsRussianDoll"
xmlns="http://langexplr.blogspot.com/DocsRussianDoll"
elementFormDefault="qualified">
<xs:element name="page">
<xs:complexType>
<xs:sequence>
<xs:element name="header">
<xs:complexType>
<xs:sequence>
<xs:element name="content" type="xs:string" />
</xs:sequence>
<xs:attribute name="margin"
type="xs:integer" />
</xs:complexType>

</xs:element>
<xs:element name="body">
<xs:complexType>
<xs:sequence>
<xs:element name="paragraph"
type="xs:string" minOccurs="0" maxOccurs="unbounded" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="footer">
<xs:complexType>
<xs:sequence>
<xs:element name="content" type="xs:string" />
</xs:sequence>
<xs:attribute name="margin"
type="xs:integer" />
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>


The Scala code to identify this design pattern looks like this:


class RussianDoll extends XsdDesignPattern {
def name = "Russian Doll"
def identify(schema : XSDSchema) =
schema match {
case XSDSchemaParts(
namespace,
List(),
List(XSDElementParts(
name,
XSDComplexType(
null,
XSDSimpleSequenceModelGroup(elements))))) => {
true
}
case _ => false
}
}




Salami Slice

This design pattern says that all elements must be declared at the top level with the type declaration inside of them.

For example:


<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
targetNamespace="http://langexplr.blogspot.com/DocsSalamiSlice"
xmlns:tns="http://langexplr.blogspot.com/DocsSalamiSlice"
xmlns="http://langexplr.blogspot.com/DocsSalamiSlice"
elementFormDefault="qualified">

<xs:element name="content" type="xs:string" />
<xs:element name="paragraph" type="xs:string" />

<xs:element name="header">
<xs:complexType>
<xs:sequence>
<xs:element ref="tns:content" />
</xs:sequence>
<xs:attribute name="margin" type="xs:integer" />
</xs:complexType>
</xs:element>

<xs:element name="footer">
<xs:complexType>
<xs:sequence>
<xs:element ref="tns:content" />
</xs:sequence>
<xs:attribute name="margin" type="xs:integer" />
</xs:complexType>
</xs:element>

<xs:element name="body">
<xs:complexType>
<xs:sequence>
<xs:element ref="tns:paragraph" minOccurs="0"
maxOccurs="unbounded" />
</xs:sequence>
</xs:complexType>
</xs:element>

<xs:element name="page">
<xs:complexType>
<xs:sequence>
<xs:element ref="tns:header" />
<xs:element ref="tns:body" />
<xs:element ref="tns:footer" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>



The Scala code to identify this design pattern looks like this:


class SalamiSlice extends XsdDesignPattern {
def name = "Salami Slice"
def identify(schema : XSDSchema) =
schema match {
case XSDSchemaParts(
namespace,
List(),
elements) => {
elementsWithReferences(elements)
}
case _ => false
}
// Utility methods

def forAllInnerElements(l : List[XSDElementDeclaration],
pred : XSDElementDeclaration => boolean) =
l.forall{
case XSDElementParts(
_,
XSDComplexType(null,XSDSimpleSequenceModelGroup(particles))) =>
particles.forall({
case XSDParticleContent(e:XSDElementDeclaration) => pred(e)
case _ => false })
case XSDElementParts(_,XSDComplexType(null,null)) => true
case XSDElementParts(_,XSDSimpleType(_)) => true
case _ => false
}

def elementsWithReferences(x : List[XSDElementDeclaration]) =
forAllInnerElements(
x,
(e:XSDElementDeclaration) => e.isElementDeclarationReference)

}




Venetian Blind

This design pattern says that there one global element and all other elements use types declared at the top level.

For example:


<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
targetNamespace="http://langexplr.blogspot.com/DocsVenetianBlind"
xmlns:tns="http://langexplr.blogspot.com/DocsVenetianBlind"
xmlns="http://langexplr.blogspot.com/DocsVenetianBlind"
elementFormDefault="qualified">

<xs:complexType name="sectionType">
<xs:sequence>
<xs:element name="content" type="xs:string" />
</xs:sequence>
<xs:attribute name="margin" type="xs:integer" />
</xs:complexType>

<xs:complexType name="bodyType">
<xs:sequence>
<xs:element name="paragraph" type="xs:string" minOccurs="0"
maxOccurs="unbounded" />
</xs:sequence>

</xs:complexType>

<xs:element name="page">
<xs:complexType>
<xs:sequence>
<xs:element name="header" type="tns:sectionType" />
<xs:element name="body" type="tns:bodyType" />
<xs:element name="footer" type="tns:sectionType" />
</xs:sequence>
</xs:complexType>
</xs:element>

</xs:schema>




The Scala code for this pattern looks like this:


class VenetianBlind extends XsdDesignPattern {
def name = "Venetian Blind"
def identify(schema : XSDSchema) =
schema match {
case XSDSchemaParts(
namespace,
types,
List(XSDElementParts(
_,
XSDComplexType(
_,
XSDSimpleSequenceModelGroup(elements))))) =>
elements.forall((e:XSDParticle) =>
elementWithTypeReferences(e,types))
case _ => false
}
def elementWithTypeReferences(e : XSDParticle, types : List[XSDTypeDefinition]) =
e match {
case XSDParticleContent(e:XSDElementDeclaration) =>
e.getTypeDefinition.getContainer.isInstanceOf[XSDSchema] &&
!(types.find ((t:XSDTypeDefinition) => t == e.getTypeDefinition)).isEmpty
case _ => false
}

}



Garden of Eden

This design pattern says that all the elements and types must be declared global.


<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
targetNamespace="http://langexplr.blogspot.com/DocsGardenOfEden"
xmlns:tns="http://langexplr.blogspot.com/DocsGardenOfEden"
xmlns="http://langexplr.blogspot.com/DocsGardenOfEden"
elementFormDefault="qualified">

<xs:complexType name="sectionType">
<xs:sequence>
<xs:element ref="tns:content" />
</xs:sequence>
<xs:attribute name="margin" type="xs:integer" />
</xs:complexType>

<xs:complexType name="bodyType">
<xs:sequence>
<xs:element ref="tns:paragraph" minOccurs="0"
maxOccurs="unbounded" />
</xs:sequence>
</xs:complexType>

<xs:element name="content" type="xs:string" />

<xs:element name="paragraph" type="xs:string"/>

<xs:element name="header" type="tns:sectionType" />

<xs:element name="body" type="tns:bodyType" />

<xs:element name="footer" type="tns:sectionType" />

<xs:complexType name="pageType">
<xs:sequence>
<xs:element ref="tns:header" />
<xs:element ref="tns:body" />
<xs:element ref="tns:footer" />
</xs:sequence>
</xs:complexType>

<xs:element name="page" type="tns:pageType" />
</xs:schema>



The Scala code for this pattern looks like this:


class GardenOfEden extends XsdDesignPattern {
def name = "Garden Of Eden"
def identify(schema : XSDSchema) =
schema match {
case XSDSchemaParts(
namespace,
types,
elements) =>
elements.forall((e : XSDElementDeclaration) =>
elementWithTypeReferences(e,types))
case _ => false
}

def elementWithTypeReferences(e : XSDElementDeclaration, types : List[XSDTypeDefinition]) =
e.getTypeDefinition.getContainer.isInstanceOf[XSDSchema] &&
((types.find ((t:XSDTypeDefinition) => t == e.getTypeDefinition)) match {
case Some(XSDComplexType(_,XSDSimpleSequenceModelGroup(particles))) =>
particles.forall({
case XSDParticleContent(e:XSDElementDeclaration) =>
e.isElementDeclarationReference
case _ => false })
case Some(XSDComplexType(_,null)) => true
case Some(XSDSimpleType(_)) => true
case None =>
e.getTypeDefinition.getTargetNamespace == "http://www.w3.org/2001/XMLSchema"
case _ => false
})

}





Finally we need a class to test all the patterns:


object XsdDesignPatterns {
def patterns:List[XsdDesignPattern] = List(new RussianDoll,
new SalamiSlice,
new VenetianBlind,
new GardenOfEden)
def identify(schema : XSDSchema) =
patterns.filter((p:XsdDesignPattern) => p identify schema).map((p:XsdDesignPattern) => p.name)
}




The code for this experiment can be found here.

Friday, May 25, 2007

Pattern matching on Java Objects with Scala Extractors

Scala extractors provide a nice way to use pattern matching on common Java objects. In this post I'm going to show a simple use of extractors with Java reflection objects.

More detailed information about Scala extractors can be found in the main website, in the Scala Language Specification section 8.1.7 and in the Matching Object With Patterns paper.

An extractor object can be used to extract simple values from Java objects. For example, the following code defines a extractor object that returns the name of a java.lang.Class.


object ClassName {
def unapply(c : java.lang.Class) =
Some(c.getName())
}


This extractor can be used as follows:



val c = "hola".getClass()

c match {
case ClassName(name) =>
System.out.println("The class name is: "+name)
}



Also extractors can be combined with other Scala pattern matching elements. For example, given the following extractor definitions:


object ClassName {
def unapply(c : java.lang.Class) =
Some(c.getName())
}

object ClassMethods {
def unapply(c : java.lang.Class) =
Some(c.getMethods())
}


object MethodParts {
def unapply(c : java.lang.reflect.Method) =
Some((c.getName(),c.getReturnType(),c.getParameterTypes()))
}


We can write:


def methodsThatReturnString(c : java.lang.Class) =
c match {
case ClassMethods(mts) =>
List.fromArray(
for {val m <- mts
m match {
case MethodParts(_,ClassName("java.lang.String"),_) => true
case _ => false
}} yield m)
}


Another example of this is the following:

Given these definitions:


object MemberClasses {
def unapply(c : java.lang.Class) =
Some(c.getClasses())
}


We can write:


Class.forName("java.util.Map") match {
case c@MemberClasses(Array(ClassName(inner))) =>
System.out.println("The class or interface "+
c.getName()+
" has one inner class/interface called "+
inner)
case _ => System.out.println("No inner classes")
}



With these simple/artificial examples we only covered the surface of what can be done with pattern matching in existing Java libraries. As with F# active patterns in .NET, Scala Extractors seems to be a useful tool to deal with complex object models.

Saturday, May 19, 2007

Using F# active patterns with LINQ expression trees

F# active patterns provide a nice way to handle complex object models.

In this post, I'm going to write a couple of active patterns to access parts of LINQ expression trees. As a little experiment I'm going to change the implementation of the Linq To Google Desktop expression tree handling from C# to F#.


The following definitions of active patterns provide access to sections of expression trees.


let (|BinaryExpression|_|) (x:Expression) =
if (x :? BinaryExpression)
then let be = (x :?> BinaryExpression)
in Some (be.Left,be.Right)
else None

let (|AndAlso|_|) (x:Expression) =
if (x.NodeType = ExpressionType.AndAlso)
then match x with
| BinaryExpression(l,r) -> Some (l,r)
| _ -> None
else None

let (|Equal|_|) (x:Expression) =
if (x.NodeType = ExpressionType.Equal)
then match x with
| BinaryExpression(l,r) -> Some (l,r)
| _ -> None
else None

let (|IsExpression|_|) (x:Expression) =
if (x :? TypeBinaryExpression)
then let be = (x :?> TypeBinaryExpression)
in Some (be.Expression,be.TypeOperand)
else None

let (|MethodCall|_|) (x:Expression) =
if (x.NodeType = ExpressionType.Call &&
(x :? MethodCallExpression))
then
let mc = x :?> MethodCallExpression
in Some (mc.Object,
mc.Method,
IEnumerable.to_list(mc.Arguments))
else None

let (|Cast|_|) (x:Expression) =
if (x.NodeType = ExpressionType.Convert
&& (x :? UnaryExpression))
then let ue = (x :?> UnaryExpression)
in Some (ue.Type,ue.Operand)
else None

let (|Lambda|_|) (x:Expression) =
if ( (x :? UnaryExpression) &&
((x :?> UnaryExpression).Operand :? LambdaExpression))
then let ue = (x :?> UnaryExpression)
in Some (ue.Operand :?> LambdaExpression).Body
else None

let (|MemberAccess|_|) (x:Expression) =
if (x.NodeType = ExpressionType.MemberAccess
&& (x :? MemberExpression))
then let ue = (x :?> MemberExpression)
in Some (ue.Expression,ue.Member)
else None

let (|MethodName|) (m:MethodInfo) = m.Name
let (|TypeName|) (t:Type) = t.Name

let (|PropertyWithName|_|) (m:MemberInfo) =
if (m :? PropertyInfo)
then Some (m.Name)
else None

let (|ExpressionType|) (x:Expression) = x.Type


As you can see, most of the active patterns are Partial Recognizers (see here), for example (|Equal|_|) . This is because of all the dynamic type tests that we need to do.


The C# code for expression tree handling presented on the Linq to Google Desktop post, is located in the GDesktop.CollectQueryInfo method. By using these new active pattern definitions the code could be rewritten in F# as follows:



let rec CollectQueryInfoFS (e:Expression, qi:GDQueryInfo) =
match e with
| Lambda(body) -> CollectQueryInfoFS (body,qi)
| AndAlso(l,r) -> CollectQueryInfoFS(l,qi);
CollectQueryInfoFS(r,qi)
| MethodCall(_,MethodName("Contains"),[argument]) ->
qi.AddTerm(GetStringFromArgument(argument))
| Equal(MemberAccess(ExpressionType(TypeName("GDFileResult")),
PropertyWithName("FileType")),
value) ->
qi.FileType <- GetStringFromArgument(value)
| Equal(MemberAccess(ExpressionType(TypeName("GDEmailResult")),
PropertyWithName(pName)),
value) ->
CollectEmailProperty(pName,GetStringFromArgument(value),qi)
| IsExpression(_,TypeName("GDFileResult")) ->
qi.ElementType <- new Nullable<GDElementType>( GDElementType.File )
| IsExpression(_,TypeName("GDEmailResult")) ->
qi.ElementType <- new Nullable<GDElementType>( GDElementType.Email )
| _ -> raise (new NotSupportedException(e.ToString()))


By using the active pattern definitions, the code is much smaller and easer to read and modify.

Right now only the expression tree handling part was translated to F#. The point where we call F# looks like this:


private GDQueryInfo ProcessWhereMethodCall(MethodCallExpression mcExpression)
{
Expression theObjectArgument = mcExpression.Arguments[0];
Expression whereExpressionTree = mcExpression.Arguments[1];

GDesktop provider =
(GDesktop)((ConstantExpression)theObjectArgument).Value;

GDQueryInfo qi = new GDQueryInfo();
//CollectQueryInfo(whereExpressionTree, qi);
Langexplr.FsharpTests.Langexplr.FsharpTests.CollectQueryInfoFS(
whereExpressionTree, qi);
return qi;

}


For future posts I'm going to try to rewrite the entire implementation.

Wednesday, May 16, 2007

Pattern matching on .NET objects with F# active patterns

One of the most interesting things about F# active patterns, is that it allow the creation pattern matching expressions on existing .NET objects . In this post I'm going to show a couple of examples of using F# pattern matching with common .NET objects.

The "Combining Total and Ad Hoc Extensible Pattern Matching in a Lightweight Language Extension" paper describes this feature and gives lots of useful and representative examples (A draft of this paper was commented here).

Active patterns are officially available starting with version 1.9.1.8, but version 1.9.1.9 was used for these examples.

One the simplest ways to start defining active patterns on .NET objects is to create one that returns the value of a property. For example, the following active pattern extracts the FullName property from System.IO.FileInfo .


let (|FileInfoFullName|) (f:FileInfo) = f.FullName


Now we can using FileInfoFullName this way:


let f = (new FileInfo("ActivePatternsTests.exe"))
in
match f with
| FileInfoFullName n ->
Console.WriteLine("File name: {0}",n)


Also active patterns can be used in conjunction with built in F# patterns for example. Given the following active pattern:


let (|FileInfoNameSections|) (f:FileInfo) =
(f.Name,f.Extension,f.FullName)


We can write:


let foo (f:FileInfo) =
match f with
| FileInfoNameSections(_,".txt",fn) -> ("Text file: "+fn)
| _ -> "No text file"


Also active patterns can be combined. For example:

Given these active pattern definitions:


let (|FileSecurity|) (f:FileInfo) = f.GetAccessControl()

let (|NtAccountOwner|) (f:FileSecurity) =
let o = f.GetOwner((typeof() :
ReifiedType< NTAccount >).result) :?> NTAccount
in o.Value


We can write:


let goo (f:FileInfo) =
match f with
| FileSecurity(NtAccountOwner("MyMachine\\auser")) -> "Yes"
| _ -> "No"


Only one kind of active patterns was used in this post, the article describe several others with more interesting characteristics.

More interesting experiments can be done with other .NET APIs. One nice example is LINQ expression trees.

Saturday, May 12, 2007

Adding support for projections to Linq to Google Desktop

I this post I'm going to show how support for projections was added to the Linq To Google Desktop experiment.

In order to support queries like:


var e10 = from t in gd
where t is GDFileResult &&
t.Contains("sql")
select ((GDFileResult)t).Location;

foreach (string s in e10) {
Console.WriteLine("The file name is: " + s);
}




We need to add support for handling the Select method. The query described above is processed by the compiler as:


gd.Where( ... ).Select( ... );


Given that the Where method processing returns an instance of GDQuery, then the processing of the Select method call is done in the GDQuery.CreateQuery method. The implementation of this method is the following:


public IQueryable<T> CreateQuery<T>(System.Linq.Expressions.Expression expression)
{
if (IsSelectMethodCall(expression))
{
MethodCallExpression methodCallExpr = (MethodCallExpression)expression;

UnaryExpression unaryQuoteExpr =
(UnaryExpression)methodCallExpr.Arguments[1];

LambdaExpression lambdaExpr =
(LambdaExpression)unaryQuoteExpr.Operand;

GDQuery query =
(GDQuery)((ConstantExpression)methodCallExpr.Arguments[0]).Value;

return query.AsEnumerable<GDResult>().Select(
(Func<GDResult,T>)lambdaExpr.Compile()).AsQueryable<T>();
}
else
{
throw new NotSupportedException("Not supported method call");
}

}


What this method does is to delegate the execution to the public static IEnumerable<TResult> Select<TSource, TResult>(this IEnumerable<TSource> source, Func<TSource, TResult> selector); method , which is the extension method fro IEnumerable. The only thing we also need to do is to compile the lambda expression that represents the filter.

This is done this way because elements in the projection section cannot be used to make the Google Desktop query more specific. This is not the case of Linq To SQL where the projection section affects the way the query is processed by the DBMS.

Now we can write queries like:


var e11 = from t in gd
where t is GDFileResult &&
t.Contains("sql")
select new { FileName = ((GDFileResult)t).Location,
Info = new FileInfo(((GDFileResult)t).Location)};

foreach (var fResult in e11)
{
Console.WriteLine(fResult.Info.Name);
}