Sunday, March 15, 2009

Support for the LookupSwitch opcode in AbcExplorationLib

The LookupSwitch AVM2 opcode is used to represent the ActionScript switch statement.

This is an interesting branch instruction because it has multiple targets. It is almost a direct translation of the switch statement because it has two parameters, a default case and an array of possible targets.

Recently support for this opcode was added to AbcExplorationLib.

Given the following ActionScript code:


var x;
for(x = 1;x < 5;x++) {
switch(x) {
case 1:
print("one");
break;
case 2:
print("two");
break;
case 3:
print("three");
break;
case 4:
print("four");
break;
}
}


We compile this file to a .abc file using the following command:


$ java -jar /opt/flex3sdk/lib/asc.jar testswitch.as

testswitch.abc, 280 bytes written


By using a little IronPython example included with AbcExplorationLib we can see how this library interprets the LookupSwitch opcode:


$ mono /opt/IronPython-2.0/ipy.exe ../ipyexample/abccontents.py testswitch.abc
...
Instructions:
getlocal_0
pushscope
pushbyte 1
getglobalscope
swap
setslot 1
jump dest177
dest12:
label
jump dest74
dest17:
label
findpropertystrict M.print
pushstring "one"
callprop M.print
pop
jump dest165
dest30:
label
findpropertystrict M.print
pushstring "two"
callprop M.print
pop
jump dest165
dest43:
label
findpropertystrict M.print
pushstring "three"
callprop M.print
pop
jump dest165
dest56:
label
findpropertystrict M.print
pushstring "four"
callprop M.print
pop
jump dest165
dest69:
label
jump dest165
dest74:
getglobalscope
getslot 1
setlocal_1
pushbyte 1
getlocal_1
ifstrictneq dest91
pushshort 0
jump dest143
dest91:
pushbyte 2
getlocal_1
ifstrictneq dest104
pushshort 1
jump dest143
dest104:
pushbyte 3
getlocal_1
ifstrictneq dest117
pushshort 2
jump dest143
dest117:
pushbyte 4
getlocal_1
ifstrictneq dest130
pushshort 3
jump dest143
dest130:
pushfalse
iffalse SolvedReference dest141
pushshort 4
jump dest143
dest141:
pushshort 4
dest143:
kill
lookupswitch dest69 dest17,dest30,dest43,dest56,dest69
dest165:
getglobalscope
getslot 1
increment
setlocal_1
getlocal_1
getglobalscope
swap
setslot 1
kill
dest177:
getglobalscope
getslot 1
pushbyte 5
iflt dest12
returnvoid


As described in the documentation the parameters of the LookupSwitch instruction are specified as relative byte offsets that specify the target. In order to give a higher level representation of the code, these relative offsets are converted to symbolic references. This process is detailed in the "Using F# Active Patterns to encapsulate complex conditions" post.

This process starts in the following functions:


static member ReadAndProcessInstructions(aInput:BinaryReader,
count,
constantPool:ConstantPoolInfo) =
let instructionsAndOffsets =
(AvmMethodBody.ReadingInstructions([],
aInput,
count,
constantPool)) in
let destinations =
AvmMethodBody.CollectDestinations(instructionsAndOffsets,
Map.empty,
instructionsAndOffsets)
in
AvmMethodBody.UpdateCodeWithDestinations(
destinations,
instructionsAndOffsets,[]) |> List.to_array


The CollectDestinations method collects all the absolute offsets used by branch instructions and stores them in a dictionary with a generated label. The UpdateCodeWithDestinations method modifies the instruction list use the generated labels.

For example to add support in CollectDestinations the following code was added:


static member CheckLookupSwitchCase baseOffset
(totalInstructions:(int64*AbcFileInstruction)
list) =
fun (destinations:Map<int64,string>) target ->
match target with
| UnSolvedReference(relativeOffset) when
(AvmMethodBody.IsDestinationDefined(int(baseOffset+relativeOffset),
totalInstructions)) ->
destinations.Add(int64(baseOffset+relativeOffset),
sprintf "dest%d" (baseOffset+relativeOffset))
| _ -> destinations

static member CollectDestinations(instructions:(int64*AbcFileInstruction) list,
destinations:Map<int64,string>,
totalInstructions:(int64*AbcFileInstruction) list) =
match instructions with
...
| ((offset,(LookupSwitch(defaultBranch,cases)))::rest) ->
let baseOffset = int(offset) in
AvmMethodBody.CollectDestinations(rest,
Seq.append [defaultBranch] cases |>
Seq.fold (AvmMethodBody.CheckLookupSwitchCase baseOffset totalInstructions ) destinations,
totalInstructions)
...


Then the code is modified in the UpdateCodeWithDestinations method to use the generated labels.


static member UpdateCodeWithDestinations(destinations:Map<int64,string>,
instructions,
resultingInstructions) =
let processedInstructions =
match instructions with
...
| ((offset,LookupSwitch(defaultCase,cases))::rest) ->
(offset,
LookupSwitch(AvmMethodBody.SolveSwitchCase defaultCase offset destinations,
Array.map (fun c->AvmMethodBody.SolveSwitchCase c offset destinations) cases))::rest
| _ -> instructions
...

Tuesday, March 3, 2009

Manipulating AVM2 byte code with F#

In this post I'm going to show an example of using AbcExplorationLib to manipulate simple AVM2 byte code (ActionScript). This example show how load a .ABC file and write it back to disk.

AbcExplorationLib is a library that will allow the manipulation of AVM2 Byte Code(described here). Although it's still incomplete, some basic examples work as the ones presented in this post .

The following ActionScript code will be compiled to byte code .


var i = 0;
for( i = 0;i < 10;i++) {
print("inside loop");
}
print("Done");


To generate the ".abc" file we type:

c:\test\> java -jar c:\flexsdk\lib\asc.jar test.as



Loading the compiled file



We're going to use the F# REPL(fsi.exe) to manipulate the file. We start by referencing the library.


> #r "abcexplorationlib.dll";;

--> Referenced 'C:\test\abcexplorationlib.dll'

> open Langexplr.Abc;;

Now we load the file:


> let abcFile = using (new System.IO.FileStream("test.abc",System.IO.FileMode.Open)) (
fun s -> AvmAbcFile.Create(s));;


Now abcFile contains the code of the compiled program.


> abcFile;;
val it : AvmAbcFile
= Langexplr.Abc.AvmAbcFile {Classes = [];
Scripts = [Langexplr.Abc.AvmScript];}




Inspecting the instructions



We're interested in the instructions of the top-level script for this .abc file. By typing the following expression we can get to this section:


> abcFile.Scripts.[0].InitMethod.Body.Value.Instructions;;
val it : AbcFileInstruction array
= [|GetLocal0; PushScope; PushByte 0uy; GetGlobalScope; Swap; SetSlot 1;
PushByte 0uy; GetGlobalScope; Swap; SetSlot 1;
Jump (SolvedReference "dest39"); ArtificialCodeBranchLabel "dest18"; Label
FindPropertyStrict
(MQualifiedName
([|Ns ("",CONSTANT_Namespace); Ns ("test.as$0",CONSTANT_PrivateNs)|],
"print")); PushString "inside loop";
CallProperty
(MQualifiedName
([|Ns ("",CONSTANT_Namespace); Ns ("test.as$0",CONSTANT_PrivateNs)|],
"print"),1); Pop; GetGlobalScope; GetSlot 1; Increment; SetLocal_2;
GetLocal2; GetGlobalScope; Swap; SetSlot 1; Kill 2;
ArtificialCodeBranchLabel "dest39"; GetGlobalScope; GetSlot 1;
PushByte 10uy; IfLt (SolvedReference "dest18");
FindPropertyStrict
(MQualifiedName
([|Ns ("",CONSTANT_Namespace); Ns ("test.as$0",CONSTANT_PrivateNs)|],
"print")); PushString "Done";
CallProperty
(MQualifiedName
([|Ns ("",CONSTANT_Namespace); Ns ("test.as$0",CONSTANT_PrivateNs)|],
"print"),1); CoerceA; SetLocal_1; GetLocal1; ReturnValue; Kill 1|]



We're going to define the following function to assist in the presentation of instruction listings.


> open Langexplr.Abc.InstructionPatterns;;
> let pr (i:AbcFileInstruction) =
- match i with
- | ArtificialCodeBranchLabel t -> printf "%s:\n" <| t.ToString()
- | i & UnsolvedSingleBranchInstruction(d,_) -> printf " %s %d\n" i.Name d
- | i & SolvedSingleBranchInstruction(l,_) -> printf " %s %s\n" i.Name l
- | _ -> printf " %s\n" i.Name;;

val pr : AbcFileInstruction -> unit


Now we can type:


> abcFile.Scripts.[0].InitMethod.Body.Value.Instructions |> Array.iter pr;;
getlocal_0
pushscope
pushbyte
getglobalscope
swap
setslot
pushbyte
getglobalscope
swap
setslot
jump dest39
dest18:
label
findpropertystrict
pushstring
callprop
pop
getglobalscope
getslot
increment
setlocal_2
getlocal_2
getglobalscope
swap
setslot
kill
dest39:
getglobalscope
getslot
pushbyte
iflt dest18
findpropertystrict
pushstring
callprop
coerce_a
setlocal_1
getlocal_1
returnvalue
kill
val it : unit = ()




A note on branch instructions



In order to make it easy to manipulate and analyze the code AbcExplorationLib adds a non-existing instruction called ArtificialCodeBranchLabel to mark the position where a branch instruction will jump. When these labels are generated the branch instructions are modified to point to the label's name instead of a relative byte offset. Details on how this process is briefly described in "Using F# Active Patterns to encapsulate complex conditions"

Converting from label references to byte offsets is also necessary to write code back to an .abc file. This process is performed by a function called ConvertSymbolicLabelsToByteReferences, for example:


> let c = AbcFileCreator();;

val c : AbcFileCreator

> abcFile.Scripts.[0].InitMethod.Body.Value.Instructions |>
- InstructionManipulation.ConvertSymbolicLabelsToByteReferences c |>
- Array.iter pr;;
getlocal_0
pushscope
pushbyte
getglobalscope
swap
setslot
pushbyte
getglobalscope
swap
setslot
jump 21
dest18:
label
findpropertystrict
pushstring
callprop
pop
getglobalscope
getslot
increment
setlocal_2
getlocal_2
getglobalscope
swap
setslot
kill
dest39:
getglobalscope
getslot
pushbyte
iflt -30
findpropertystrict
pushstring
callprop
coerce_a
setlocal_1
getlocal_1
returnvalue
kill
val it : unit = ()
>



Modifying the code



Values for branch instruction targets are adjusted if new code added, for example, lets add some code to print "Hola!" inside the loop.


> let printName = CQualifiedName(Ns("",NamespaceKind.CONSTANT_Namespace),"print"
- ) ;;

val printName : QualifiedName

> let printCode = [| FindPropertyStrict printName ;
- PushString "Hola!" ;
- CallProperty(printName,1);
- Pop |] ;;

val printCode : AbcFileInstruction array

> Seq.append instr.[0..16] <| Seq.append printCode instr.[17..] |>
- Seq.to_array |>
- InstructionManipulation.ConvertSymbolicLabelsToByteReferences c |>
- Array.iter pr;;
getlocal_0
pushscope
pushbyte
getglobalscope
swap
setslot
pushbyte
getglobalscope
swap
setslot
jump 29
dest18:
label
findpropertystrict
pushstring
callprop
pop
findpropertystrict
pushstring
callprop
pop

getglobalscope
getslot
increment
setlocal_2
getlocal_2
getglobalscope
swap
setslot
kill
dest39:
getglobalscope
getslot
pushbyte
iflt -38
findpropertystrict
pushstring
callprop
coerce_a
setlocal_1
getlocal_1
returnvalue
kill
val it : unit = ()



Writing the new file



We can write this code back to a .abc file by doing this:


> let newCode = Seq.append instr.[0..16] <| Seq.append printCode instr.[17..] |
- > Seq.to_array;;

val newCode : AbcFileInstruction array

> let newBody = AvmMethodBody(oldbody.Method,
- oldbody.MaxStack,
- oldbody.LocalCount,
- oldbody.InitScopeDepth,
- oldbody.MaxScopeDepth,
- newCode,
- oldbody.Exceptions,
- oldbody.Traits);;

val newBody : AvmMethodBody

> let newFile = AvmAbcFile( [AvmScript( abcFile.Scripts.[0].InitMethod.CloneWithBody(newBody), abcFile.Scripts.[0].Members)], []);;

val newFile : AvmAbcFile
> open System.IO;;
> let c = AbcFileCreator();;

val c : AbcFileCreator

>
- using (new BinaryWriter(new FileStream("test_modified.abc",FileMode.Create)))
-
- (fun f -> let file = newFile.ToLowerIr(c) in file.WriteTo(f));;
val it : unit = ()


Running this program using Tamarin shows:


c:\test\>avmplus_sd.exe test_modified.abc
inside loop
Hola!
inside loop
Hola!
inside loop
Hola!
inside loop
Hola!
inside loop
Hola!
inside loop
Hola!
inside loop
Hola!
inside loop
Hola!
inside loop
Hola!
inside loop
Hola!
Done



The AbcExplorationLib library is still pretty incomplete. Also there's a lot to improve, for example name handling and instruction modification. Future posts will present new features/experiments.

Thursday, February 12, 2009

Writing Xml with IronPython, XmlWriter and the 'with' statement

This post shows a little example of wrapping calls to System.Xml.XmlWriter inside a Python's 'with' statement using IronPython.

Writing Xml



While reading some code examples from the XIST HTML/XML generation library I noticed the nice use of Python's 'with' statement to represent the target HTML or XML.

The System.Xml.XmlWriter class provided with .NET already gives you a way to write well formed Xml documents. In this post I'm going to show how to use an XmlWriter instance in conjunction with Python's 'with' statement.

We want to write the following code:


from __future__ import with_statement

...

w = XmlWriter.Create(System.Console.Out,XmlWriterSettings(Indent=True))
x = XWriter(w)

with x.element('tableofcontents'):
with x.element('section',{'page' : '10'}):
x.text('Introduction')
with x.element('section',{'page' : '12'}):
x.text('Main topic')
with x.element('section',{'page' : '14'}):
x.text('Extra topic')


To generate the following Xml file:


<tableofcontents>
<section page="10">Introduction</section>
<section page="12">Main topic</section>
<section page="14">Extra topic</section>
</tableofcontents>


The 'with' statement



The 'with' statement was introduced in Python 2.5 . This statement is used wrap the execution of a series of statements with some special code. For example it is used to implement the try...except...finally pattern.

As described in the documentation the following statement:


with context expression:
statements...


Will be executed as follows:


  1. Evaluate the context expression to obtain the context manager

  2. Invoke the context manager's __enter__() method

  3. Execute the statements

  4. When the execution of the statements finishes(even with an exception), the context manager's __exit()__ method is called.



Given these steps we're going to implement a context manager that assist in the creation of Xml documents using the System.Xml.XmlWriter .NET class.

The following code shows a class that wraps the XmlWriter instance and helps with the creation of context managers:


class XWriter(object):
def __init__(self,writer):
self.writer = writer

def element(self,name,atts = {}):
return ElementCtxt(name,atts,self)

def nselement(self,prefix,name,namespace,atts = {}):
return NamespaceElementCtxt(prefix,name,namespace,atts,self)


def text(self,text):
self.writer.WriteString(text)

def cdata(self,text):
self.writer.WriteCData(text)



Notice that the element method creates an instance of the ElementCtxt class using the element name and an optional dictionary with the attributes. As the following listing shows this class performs the calls to WriteStartElement and WriteEndElement in the __enter__ and __exit__ methods.


class ElementCtxt(object):
def __init__(self,elementName,atts,writer):
self.elementName = elementName
self.atts = atts
self.writer = writer

def processAttributes(self):
for att in self.atts:
self.writer.writer.WriteAttributeString(att,self.atts[att].__str__())

def processStartTag(self):
self.writer.writer.WriteStartElement(self.elementName)
self.processAttributes()

def __enter__(self):
self.processStartTag()
return self

def __exit__(self,t,v,tr):
self.writer.writer.WriteEndElement()
return t == None



The XWriter.nselement method is used to write elements with namespace and prefix. This call generates an instance of the following context manager:


class NamespaceElementCtxt(ElementCtxt):
def __init__(self,prefix,elementName,namespace,atts,writer):
ElementCtxt.__init__(self,elementName,atts,writer)
self.namespace = namespace
self.prefix = prefix
def processStartTag(self):
self.writer.writer.WriteStartElement(self.prefix,self.elementName,self.namespace)
self.processAttributes()


Final example



The following code shows how to create a little SVG file:


from __future__ import with_statement
from xmlwriterw import XWriter

import clr

clr.AddReference('System.Xml')

from System.Xml import *
import System


w = XmlWriter.Create(System.Console.Out,\
XmlWriterSettings(Indent=True))
x = XWriter(w)

svgNs = 'http://www.w3.org/2000/svg'

with x.nselement('s','svg',svgNs,{'version': '1.1',
'viewBox': '0 0 100 100',
'style':'width:100%; height:100%; position:absolute; top:0; left:0; z-index:-1;'}):
with x.nselement('s','linearGradient',svgNs, { 'id' : 'gradient' }):
with x.nselement('s','stop',svgNs, {'class' : 'begin',
'offset' : '0%',
'stop-color':'red'}):
pass
with x.nselement('s','stop',svgNs, {'class' : 'end',
'offset' : '100%'}):
pass
with x.nselement('s','rect',svgNs, { 'x':0,
'y':0,
'width':100,
'height':100,
'style':'fill:url(#gradient)'} ):
pass
for i in range(1,5):
with x.nselement('s','circle',svgNs, { 'cx': 50,
'cy': 50,
'r': 30 - i*3,
'style':'fill:url(#gradient)'} ):
pass

w.Close()


Running this program shows:


<s:svg viewBox="0 0 100 100" style="width:100%; height:100%; position:absolute; top:0; left:0; z-index:-1;" version="1.1" xmlns:s="http://www.w3.org/2000/svg">
<s:linearGradient id="gradient">
<s:stop offset="0%" class="begin" stop-color="red" />
<s:stop offset="100%" class="end" />
</s:linearGradient>
<s:rect x="0" height="100" width="100" style="fill:url(#gradient)" y="0" />
<s:circle cx="50" cy="50" style="fill:url(#gradient)" r="27" />
<s:circle cx="50" cy="50" style="fill:url(#gradient)" r="24" />
<s:circle cx="50" cy="50" style="fill:url(#gradient)" r="21" />
<s:circle cx="50" cy="50" style="fill:url(#gradient)" r="18" />
</s:svg>

Wednesday, January 21, 2009

Some notes on using F# code from IronPython

This post presents a couple of things I learned about the interaction between IronPython and F#.

Most information about this topic can be found by looking at the code generated by the F# compiler, using tools like ILDASM or Reflector. Also the F# from C# section of the F# 1.1.12 documentation is very helpful.

FSharp.Code library



In order to call some F# basic functionality we need to add a reference to FSharp.Core.

Python

import clr
clr.AddReference('FSharp.Core')



Type parameters



IronPython provides a way to specify type parameters to methods and classes by using square brackets followed by the names of the types to be used. This mechanism can be used to create F# constructs that require them.

The following example shows how to create an F# list:

Python

>>> import clr
>>> clr.AddReference('FSharp.Core')
>>> from Microsoft.FSharp.Collections import List
>>> l1 = List[str].Cons('World',List[str].Empty)
>>> l2 = List[str].Cons('Hello',l1)
>>> " ".join(l2)
'Hello World'



Type parameters, for methods that require them, also can be specified using the same syntax. For example the following code shows the use of Seq.length with an Python sequence.

Python

print Seq.length[int]([45,234,52,345,2,346,657])



Tuples



Tuples in F# are instances of the Microsoft.FSharp.Core.Tuple class (with several definitions with depending on the number of arguments). A tuple instance contains properties Item# where # is the position of the element.

The following example shows how to use a tuple created in F# from IronPython.

F#

module MyFunctions = begin
...
let numberEvenTest x = x,x % 2 = 0
...
end


Python

print MyFunctions.numberEvenTest(46)
print MyFunctions.numberEvenTest(46).Item1
print MyFunctions.numberEvenTest(46).Item2



This program will print:


(46,True)
46
True


To create a tuple inside IronPython the Tuple[type1,type2,...](value1,value2,...) constructor must be used.

For example to given the following definition:

F#

module MyFunctions = begin
...
let movePoints points xdistance ydistance =
Seq.map (fun (x,y) -> (x+xdistance,y+ydistance)) points
...
end


We can used from IronPython like this:
Python

from Microsoft.FSharp.Core import Tuple
...
points = [ Tuple[int,int](1,2), Tuple[int,int](5,3) ]

for p in MyFunctions.movePoints(points,20,40):
print p


It is tempting to use the Python syntax for assigning a list of variables to the values of the tuples. For example:


x,y,z = 2,3,5


However this is not possible since F# tuples are not compatible with IronPython sequences.

Operators



F# operators are created using the same conventions as the C# operators. Thus they can be used directly in IronPython.

For example the following F# definition:


type Complex(real:double,img:double) = class
member this.real with get() = real
member this.img with get() = img
static member (+) (c1:Complex,c2:Complex) =
Complex(c1.real+c2.real,c1.img+c2.img)
override this.ToString() = sprintf "%f + %fi" real img
end


Can be used in IronPython:

Python


c1 = Complex(20,234)
c2 = Complex(34.35,32.2)
c3 = (c1+c2)



Discriminated unions



Discriminated unions in F# provide a compact way to define data structures. The following example shows the definition of simple math expressions that include addition, subtraction and numeric literals.

F#

type MathExpr =
| Addition of MathExpr * MathExpr
| Subtraction of MathExpr * MathExpr
| Literal of double


The following code snippet shows how to create instance of these math expression elements from IronPython.

Python

expr = MathExpr.Addition(MathExpr.Literal(10),\
MathExpr.Subtraction(\
MathExpr.Literal(40), \
MathExpr.Literal(24)))


A instance of the discriminated union has methods IsXXXX to verify the kind. Also properties a generated with the name of the constructor followed by a number (for example Addition1) to access values given to the constructor.

The following example shows how to evaluate a math expression.

Python

def evalExpr(expr):
if expr.IsAddition():
return evalExpr(expr.Addition1) + evalExpr(expr.Addition2)
elif expr.IsSubtraction():
return evalExpr(expr.Subtraction1) - evalExpr(expr.Subtraction2)
else:
return expr.Literal1


Functions



Functions passed as parameters need to be converted to a special F# element that represent them. In order to do this the FuncConvert.ToFastFunc function is used.

For example the following code shows how to call Seq.iter to print all the elements of a Python list.


from Microsoft.FSharp.Core import FuncConvert
...
aList = [5,3,42,6]

def foo(x):
print(x)

Seq.iter[int](FuncConvert.ToFastFunc[int]( foo),aList)



Since IronPython and F# use IEnumerable<T> types to represent collections and sequences we can use F# functions to manipulate IronPython collections.

For example given the following definition of a Python generator for Fibonacci numbers:


def fibo():
last1 = 0
last2 = 1
while True:
yield last1
next = last1+last2
last2 = last1
last1 = next


We can call F# functions to process values generated by fibo. For example the following IronPython code prints the first 15 squared Fibonacci numbers.



for n in Seq.map[int,int]( FuncConvert.ToFastFunc[int,int](lambda x: x*x) ,\
Seq.take[int](15,fibo())):
print n




Code for this post was created using IronPython 2.0 and F# 1.9.6.2 .

Monday, December 1, 2008

A quick look at MEF with F#

In this post I'm going to show a little example of using the Managed Extensibility Framework (MEF) from F#.

The Managed Extensibility Framework (MEF) is a very interesting framework for building extensible applications. It is planned to be used in products such as Visual Studio 2010.

Episode .NET Rocks #398: Glenn Block on MEF is a nice source of information the project.

The MEF Codeplex page has documentation on how to start using it. Also the Addicted to MEF series is a nice source of information on how to start using it.

The example



For this example I want to define a little program F# program that plots math functions. Function definitions will be represented as classes that implement the IFunction interface. These definitions will be taken from DLLs stored in a "extensions" directory.

Here's the definition of the IFunction interface:


type IFunction = interface
abstract Apply : double -> double
end



The following code shows the definition of the Winforms Form that is used to display the function:


open System
open System.Windows.Forms
open System.Drawing
open FunctionDefinitions
open System.ComponentModel.Composition
open System.Collections.Generic

type MyForm() as this = class
inherit System.Windows.Forms.Form()

let mutable functions : IEnumerable<IFunction> = Seq.empty

do this.Size <- new System.Drawing.Size(300,300)
do this.BackColor <- Color.White
do this.Text <- "Functions"
do this.Paint.Add(
fun (pargs:PaintEventArgs) ->
using(new Pen(Brushes.Black))
(fun(p) -> pargs.Graphics.DrawLine(p,
new Point(this.Size.Width/2,0),
new Point(this.Size.Width/2,this.Size.Height))

pargs.Graphics.DrawLine(p,
new Point(0,this.Size.Height/2),
new Point(this.Size.Width,this.Size.Height/2))
Seq.iter (fun func -> pargs.Graphics.DrawLines(p,this.CalcPoints(func))) functions
))

member this.CalcPoints(func:IFunction) : Point array =
{ 0..300} |> Seq.map (fun x -> -1.0/30.0 * double(x) + 5.0) |>
Seq.map (fun x -> x,func.Apply(x)) |>
Seq.map (fun(x,y) -> new Point(int32(x*30.0+150.0),
int32(y*(-30.0)+150.0))) |>
Seq.to_array

[<Import>]
member this.Functions
with get() : IEnumerable<IFunction> = functions
and set (f : IEnumerable<IFunction>) = functions <- f

end


In order to say that we want all available functions(IFunction) identified by MEF we declare the Functions property with type IEnumerable<IFunction> and decorated with the Import attribute.

The following code shows the how we use MEF to compose the application and run the winforms app:


let f = new MyForm()
let catalog = new DirectoryPartCatalog("c:\\temp\\Extensions")
let container = new CompositionContainer(catalog,[||])

container.AddPart(f)
container.Compose()

System.Windows.Forms.Application.Run(f)


Notice that by using DirectoryPartCatalog we say that we want to take all extensions(math function definitions) stored in c:\temp\Extensions.

It is important to notice that the program that has the Winforms application is stored in a different assembly from where IFunction is defined. The reason is that the extension DLLs need a reference to it.

Running this program without extensions shows the following form:




The following code shows an extension written in F#:


#light

namespace MoreFunctions
open FunctionDefinitions
open System.ComponentModel.Composition

[<Export(typeof<IFunction>)>]
type AFunction() = class
interface IFunction with
member this.Apply(x:double) = System.Math.Sin(x)
end


We compile this file and copy the result to the extensions directory.


fsc.exe other.fs -r FunctionDefinitions.dll -r System.ComponentModel.Composition.dll -a
copy other.dll c:\temp\Extensions


By running the program again the following form is shown




Since we're working with common .NET assemblies we can write a function definition using C#, for example:



using FunctionDefinitions;
using System.ComponentModel.Composition;
namespace Test2 {
[Export(typeof(IFunction))]
public class AFunction : IFunction
{
public double Apply(double d)
{
return d*d;
}
}
}


We compile this file and copy it to the extensions directory:


csc /reference:FunctionDefinitions.dll;System.ComponentModel.Composition.dll /t:library AFunc.cs
copy AFunc.dll c:\temp\Extensions



Running this program shows the following form:




Final words



MEF provides a nice/simple way to add extensions to .NET applications. I was really impressed by how simple it is to define Exports and Imports and how little MEF-specific code you have to write in order to make it work.

One thing that will be interesting to see is how this framework is going to interact with things such as the DLR. A nice example of this is Intellipad which, as mentioned in the .NET Rocks interview, is used to allow IronPython extensions.

Code for this post uses MEF Preview 3 release.

Friday, November 21, 2008

Using the Canvas Html element

In this post I'm going to show a little example of using the HTML 5 Canvas element.

The Canvas element provides a way to draw graphics using Javascript. It is currently supported in browsers such as Safari, Firefox, Opera and Chrome. Sadly it is not supported in IE.

There's a lot of documentation on how to use this element. The Mozilla Developer Center provides a nice tutorial: Canvas tutorial. Also the HTML 5 specification document provides a detailed description of the element in The canvas element section.

The Example



In order to play learn about this element I decided to create a toy program that takes an image and creates a little Jigsaw Puzzle of it.

Here's how the page looks:



The running version of this program can be found here.

The Code



The code to create this little program is very simple. First the canvas element is specified as follows:


...
<body onload="init();" >
<p>Canvas Object Tests</p>
<canvas id="myCanvas" width="700" height="600"
onmousemove="moveHandler(event);"
onmousedown="mouseDownHandler(event);"
onmouseup="mouseUpHandler(event);"
onmouseout="mouseUpHandler(event);"></canvas>
<hr/>
<div id="notSup" ></div>

</body>
...


Four event handlers are added to the canvas element to handle mouse interactions to move the pieces of the puzzle.

The init function is called in the "load" event of the body element.


var imageName = 'http://farm3.static.flickr.com/2161/2543461722_a68f3d1e2e_m.jpg';
...
function init() {
img = new Image();

img.onload = function() {
imageLoaded = true;
draw();
}
img.src = imageName;

theObjects = createImageSegments(3,3,{width:240,height:180},{x:5,y:10});

draw();
}


The createImageSegments function creates the pieces of the puzzle.


function createImageSegments(lines,cols,imageSize,basePosition)
{
var incrY = imageSize.height/lines;
var incrX = imageSize.width/cols;
var theX = basePosition.x;
var theY = basePosition.y;
var result = new Array();
for(var i = 0;i < lines;i++) {
for(var j = 0;j < cols;j++) {
result.push(new ImageSegment(this.img,
{x:theX,y:theY},
{width:incrX,height:incrY},
{x:(j*incrX),y:(i*incrY)},
{width:incrX,height:incrY}));
theY += 2 + incrY;
}
}
return shuffleArray(result);
}


The ImageSegment objects are defined by the following function:


function ImageSegment(image,position,size,insidePoint,insideSize){
this.image = image;
this.position = position;
this.size = size;
this.insidePoint = insidePoint;
this.insideSize = insideSize;

this.draw = function(context) {
context.drawImage(
this.image,
this.insidePoint.x,
this.insidePoint.y,
this.insideSize.width,
this.insideSize.height,
this.position.x,
this.position.y,
this.size.width,this.size.height);
};

this.isInside = function(point) {
return (this.position.x <= point.x &&
this.position.y <= point.y &&
this.position.x + this.size.width >= point.x &&
this.position.y + this.size.height >= point.y);
}
}



Two methods are defined: one to draw the section of the image and one to determine of a given point hits the current object.

In order to draw the image the following function is defined:


function draw() {
var myCanvas = document.getElementById("myCanvas");
if(myCanvas.getContext) {

var ctxt = myCanvas.getContext('2d');
ctxt.clearRect(0,0,myCanvas.width,myCanvas.height);
if (imageLoaded) {
for(var i = 0;i < theObjects.length;i++) {
theObjects[i].draw(ctxt);
}
}

// Draw the frame
...


In order to allow the user to drag the pieces of the puzzle across the canvas the following mouse event handlers were added to the canvas.


function mouseDownHandler(e)
{
var x = e.pageX - e.target.offsetLeft;
var y = e.pageY - e.target.offsetTop;
for(var i = 0; i < theObjects.length;i++) {
var theObject = theObjects[i];
if (theObject.isInside({x:x,y:y})) {
dragging = true;
lastPoint.x = x;
lastPoint.y = y;
currentDragObject = theObject;
}
}
}

function mouseUpHandler(e)
{
dragging = false;
lastPoint.x = -1;
lastPoint.y = -1;
}

function moveHandler(e) {
if (dragging) {
var x = e.pageX - e.target.offsetLeft;
var y = e.pageY - e.target.offsetTop;
var deltaX = x - lastPoint.x;
var deltaY = y - lastPoint.y;


currentDragObject.position.x += deltaX;
currentDragObject.position.y += deltaY;


lastPoint.x = x;
lastPoint.y = y;
draw();
}
}



Basically what these event handlers do is to calculate the new position of the image being dragged and update the canvas.

Conclusion




The Html Canvas object provides a nice way to draw custom graphics in a web page using Javascript. For future posts I'm going to try to rewrite the example from the
JavaFX Script, Silverlight and Flex posts using Javascript and Canvas.

Thursday, November 13, 2008

Using a MGrammar parser from F#

In this post I'm going to show a little example of manipulating the output tree of a MGrammar parser using some F# features.

MGrammar



MGrammar is one of the components of the "Oslo" Modeling Platform which allows the definition of domain specific languages.

In his post I'm not giving an introduction to MGrammar. There are really nice articles about it for example: MGrammar in a Nutshell, Parsing with Oslo’s MGrammar (Mg) and Creating a WatiN DSL using MGrammar. Also the Oslo:Building Textual DSLs talk provides a very nice introduction to the topic.

The example



For this post I'm going to create a MGrammar definition of a little language that describes the structure of a binary file. Then I'm going to create a F# program that interprets these definitions to load binary files.

The following code shows an example of the language to be defined:


test = begin
byte == 0x22
int32 x
int16 w
bytes[16] k
end


This code describes binary files that start with a byte with the 0x22 value, followed by a int32 value associated with the "x" name, followed by an int16 value named "w" followed by an array of 16 bytes named "k".

The grammar



The grammar for this language is the following:


module LangexplTests
{
language BinaryRecognizer
{
token Int32Type = "int32";
token Int16Type = "int16";
token ByteType = "byte";

token Bytes = "bytes";

token Integer = ("0".."9")+;
token HexNumber = "0x" ("0".."9"|"A".."F")+;
token Symbol = (("A".."Z") | ("a".."z"))+;

token NewLine = "\n";
token LineFeed = "\r";
token LineBreak = LineFeed? NewLine;

token Begin = "begin";
token End = "end";

syntax TypeName = Int32Type => Int32[] |Int16Type => Int16[] |ByteType => Byte[];

syntax Expression = i:Integer=>Int[i] | s:Symbol => Var[s] | h:HexNumber => Hex[h];

syntax TypeDeclaration = TypeName Symbol;

syntax MultipleBytesDeclaration = Bytes "[" n:Expression "]" s:Symbol =>
MultipleBytesDeclaration[n,s];
syntax LiteralValueDeclaration = t:TypeName "==" e:Expression => LiteralValueDeclaration[t,e];

syntax ItemDeclaration = TypeDeclaration|MultipleBytesDeclaration|LiteralValueDeclaration;

syntax Sequence(G,Separator) =
e:G => [e]
| es:Sequence(G,Separator) Separator e:G => [valuesof(es),e];


syntax Definition = name:Symbol "=" Begin LineBreak
decs:Sequence(ItemDeclaration, LineBreak)
LineBreak
End
LineBreak* => Recognizer[name,decs];

syntax Main = Definition;

interleave Whitespace = " ";
}
}


Parsing the code



After compiling the grammar we can now load it in F#. Based on the example presented the in the introductory articles we can write:


open Microsoft.M.Grammar
open System.Dataflow

...

let runProgram() =
printf "Start\n"
let parser = MGrammarCompiler.LoadParserFromMgx(basepath+"BinaryFileRecognizer.mgx", null);
let parsedDocument = parser.ParseObject(basepath+"sample.brg",ErrorReporter.Standard)
let reco = buildRecognizer(parsedDocument)
...


The buildRecognizer will navigate the MGraph structure generated by the parser and generate a binary file recognizer based on the code from the "Using F# computation expressions to read binary files" post.

Active patterns for the parsed AST



As described in the C# samples provided in the "Programatic" section of the MGrammar in a Nutshell article, the GraphBuilder class provides a way to access parts of the parsed AST.

By defining a some F# Active Patterns we can improve the experience of navigating this tree structure. For example:


let (|SequenceElements|_|)(x) =
let gb = new GraphBuilder()
in
if (gb.IsSequence(x)) then
Some(Seq.to_list <| gb.GetSequenceElements(x))
else
None


let (|Entity|_|)(x) =
let gb = new GraphBuilder()
in
if (gb.IsEntity(x)) then
Some(gb.GetEntityLabel(x),
gb.GetEntityMembers(x)
|> Seq.map (fun (kvp:System.Collections.Generic.KeyValuePair) -> kvp.Value)
|> Seq.to_list)
else
None


let (|EntityName|_|)(x) =
let gb = new GraphBuilder()
in
if (gb.IsEntity(x)) then
Some(gb.GetEntityLabel(x))
else
None

let (|Identifier|_|)(x:obj) =
if (x :? System.Dataflow.Identifier) then
let identifier = (x :?> System.Dataflow.Identifier)
in
Some(identifier.Text)
else
None

let (|AString|_|)(x:obj) =
if (x :? System.String) then
Some(x :?> System.String)
else
None


These active pattern definitions allow the extraction of parts of a MGraph structure.

For example given following code:


test = begin
int32 x
end


The following tree structure is created by the parser.


Main[
Recognizer[
"test",
[
ItemDeclaration[
TypeDeclaration[
Int32[
],
"x"
]
]
]
]
]



Based on this example we can look at the definition of the buildRecognizer function that was referenced above.


let buildRecognizer(ast:obj) =
match ast with
| Entity(Identifier("Main"),
[Entity(Identifier("Recognizer"),
[AString(name);
SequenceElements(definitions)])])
->
printf "Processing %s \n" name
buildingRecognizer(definitions,Map.empty,new BinParserBuilder())
| _ -> raise (new System.InvalidOperationException("Invalid document"))


Notice that by combining the Entity and SequenceElements active patterns we can easily get the sequence of definitions for a given file.


In the buildingRecognizer we can process each different case of definitions specified in the grammar. For example for the following code:


int32 x


We get the following tree structure:


ItemDeclaration[
TypeDeclaration[
Int32[
],
"x"
]
]


And the case that processes this definition is the following:


let rec buildingRecognizer(definitions:obj list,variables: Map<int64,string>,builder:BinParserBuilder) =
match definitions with
| ((Entity(Identifier("ItemDeclaration"),[declaration]))::rest) ->
...
match declaration with
| Entity(Identifier("TypeDeclaration"),
[EntityName(Identifier(typeName));AString(name)])
->
let restParser = buildingRecognizer(rest,variables,builder)
in builder.Bind( (match typeName with
| "Int32" -> BRInt(name)
| "Int16" -> BRShort(name)
| "Byte" -> BRByte(name)
| _ -> raise(new System.InvalidOperationException("Unknown type"))), fun _ -> restParser)
...


In another example, for the following input code:


bytes[a] content


We get the following tree:


MultipleBytesDeclaration[
Var[
"a"
],
"content"
]


Which is processed by the following case:


| Entity(Identifier("MultipleBytesDeclaration"),
[Entity(Identifier("Var"),[AString(lengthVarName)]);
AString(varName)]) ->

let restParser = buildingRecognizer(rest,variables,builder)
in builder.Bind(
BoundFixedByteSequence(varName,lengthVarName,RByte), fun _ -> restParser)



Reading a BMP file



The following code shows a simple recognizer for reading 8 bit BMP files.


bmpFile = begin
byte == 0x42
byte == 0x4D
int32 fileSize
int16 resF
int16 resS
int32 pixelOffset
int32 headerSize
int32 width
int32 height
int16 colorPlanes
int16 == 8
int32 compression
int32 imageSize
int32 hResolution
int32 vResolution
int32 == 0x0
int32 == 0x0
bytes[1024] paletteColors
bytes[imageSize] bitmapData
end


By running this programming with a 32x32 8bit bmp file we get the following output (by printing the values of the variables):


Start
Parsed
Processing bmpFile
bitmapData = Bytes ff , ff , ff , ff ...
colorPlanes = 1
compression = 0
fileSize = 2102
hResolution = 0
headerSize = 40
height = 32
imageSize = 1024
paletteColors = Bytes 0 , 0 , 0 , 0 ...
pixelOffset = 1078
resF = 0
resS = 0
vResolution = 0
width = 32
2102


Final words



The use of Active Patterns can simplify the manipulation of the default AST generated by a MGrammar parser.

Something bad about this implementation is that it requires the creation of a GraphBuilder instance for each use of the active pattern. Maybe this could be solved by creating a single GraphBuilder instance for the module where the active patterns are defined.

Code for this post can be found here.