Code for this post was created using Visual Studio 2010 Beta 1, IronRuby for .NET 4 beta 1 and IronPython 2.6 beta 4 for .NET 4.
JSON.NET
For this post I'm using the JSON.NET library for loading the JSON data. This library provides several ways of loading JSON data. Here I'll be using a set of predefined classes: JObject for JSON objects, JArray for arrays, JValue for literal values, etc. All these classes inherit from JToken.
Code in this post use the JSON data returned by the Twitter REST API. An example of this data:
[
{"in_reply_to_screen_name":null,
"text":"...",
"user": { "following":null,
"description":"...",
"screen_name":"...",
"utc_offset":0,
"followers_count":10,
"time_zone":"...",
"statuses_count":155,
"created_at":"...",
"friends_count":1,
"url":"...",
"name":"...",
"notifications":null,
"protected":false,
"verified":false,
"favourites_count":0,
"location":"...",
"id": ...,
...
},
"truncated":false,
"created_at":"...",
"in_reply_to_status_id":null,
"in_reply_to_user_id":null,
"favorited":false,
"id":...,
"source":"...."
},
...
]
Dynamic queries on JSON data
Finders will be implemented for JSON arrays. As with Rail's dynamic finders the names of the required fields will be encoded in the name of the invoked method.
The following C# 4.0 code shows an example of this wrapper class in conjunction with the dynamic keyword.
JsonTextReader reader = new JsonTextReader(rdr);
JsonSerializer serializer = new JsonSerializer();
JArray o = (JArray)serializer.Deserialize(reader);
dynamic dArray = new FSDynArrayWrapper(o);
string name = "ldfallas";
foreach (var aJObject in dArray.FindAllByFavoritedAlsoByUserWithScreen_Name("false",name))
{
dynamic tobj = new FSDynJObjectWrapper(aJObject);
Console.WriteLine("========");
Console.WriteLine(tobj.user.screen_name);
Console.Write("\t'{0}'",tobj.text);
}
A small definition of the syntax used for names is the following.
method-name = "FindAllBy" ,
property-name , ("With", property-name)? ,
("AlsoBy" property-name , ("With", property-name)? ) *
property-name = valid json property name
In order to be more flexible the following syntax will also be allowed:
method-name = "find_all_by_" ,
property-name , ("_with_", property-name)? ,
("_also_by" property-name , ("_with_", property-name)? ) *
property-name = valid json property name
A sample name for this query methods look like this:
array.FindAllByFavoritedAlsoByUserWithScreen_Name("true","ldfallas")
This method will accept two parameters and is going to :
Find all the object elements from the array that has a 'Favorited' property equal to 'true' and also has an object with a 'User' property associated with an object which has a 'Screen_Name' property which is equal to the 'ldfallas'
Interoperability
One of the nice things of using the DLR infrastructure to create this feature, is that it can be used by other DLR languages. The following example is an IronRuby snippet:
require 'FsDlrJsonExperiments.dll'
include Langexplr::Experiments
while true do
print "Another try\n"
str = System::Net::WebClient.new().download_string("http://twitter.com/statuses/public_timeline.json")
json = FSDynArrayWrapper.CreateFromReader(System::IO::StringReader.new(str))
for i in json.find_all_by_user_with_time_zone('Central America') do
print i.to_string()
end
sleep(5)
end
The following IronPython code shows a little example of this wrapper class.
fReader = StreamReader(GetTwitterPublicTimeline())
jReader = JsonTextReader(fReader)
serializer = JsonSerializer()
json = FSDynArrayWrapper( serializer.Deserialize(jReader) )
for i in json.FindAllByFavoritedAlsoByUserWithScreen_Name("false","ldfallas"):
print i
Implementation
In order to implement the functionality presented here, the IDynamicMetaObjectProvider interface and the DynamicMetaObject class were used. By using these we can generate the code for the call site as a expression tree. For more information on how to use this interface see
Getting Started with the DLR as a Library Author document (available here) .
The code generated to do the filtering is an expression which uses the
Where method from System.Linq.Enumerable. The generated expression written in source using a pseudo C# looks like this:
{
object tmp;
array.Where(c => (((c Is JObject) &&
CompareHelper(
GetJObjectPropertyCI(((JObject)c), "Favorited"),
"true"))
&&
((((tmp = GetJObjectPropertyCI(((JObject)c), "User")) As JObject) != null) &&
CompareHelper(
GetJObjectPropertyCI(((JObject)tmp), "Screen_Name"),
"ldfallas"))))
}
Where
GetJObjectPropertyCI is a helper method that gets a property from a JObject by case-intensive name . And CompareHelper is a helper method to do the comparison.The implementation for
FSDynArrayWrapper was written in F#. Mainly because it's a nice language to implement this kind of features. However there's no easy way to consume this feature using F# since it doesn't use the DLR.Here's the definition:
type FSDynArrayWrapper(a:JArray) =
member this.array with get() = a
static member CreateFromReader(stream : System.IO.TextReader) =
...
static member CreateFromFile(fileName:string) =
...
interface IDynamicMetaObjectProvider with
member this.GetMetaObject( parameter : Expression) : DynamicMetaObject =
FSDynArrayWrapperMetaObject(parameter,this) :> DynamicMetaObject
As you can see, the interesting part is in the implementation of
FSDynArrayWrapperMetaObject. The CreateFromReader and CreateFromFile methods are only utility methods to load data from a document. The implementation of
FSDynArrayWrapperMetaObject looks like this:
type FSDynArrayWrapperMetaObject(expression : Expression, value: System.Object) =
inherit DynamicMetaObject(expression,BindingRestrictions.Empty,value)
...
override this.BindInvokeMember(binder : InvokeMemberBinder, args: DynamicMetaObject array) =
match QueryInfo.GetQueryElements(binder.Name) with
| Some( elements ) ->
(new DynamicMetaObject(
this.GenerateCodeForBinder(
elements,
Array.map
(fun (v:DynamicMetaObject) ->
Expression.Constant(v.Value.ToString()) :> Expression) args),
binder.FallbackInvokeMember(this,args).Restrictions))
| None -> base.BindInvokeMember(binder,args)
The
BindInvokeMember creates the expression tree for the code that will be executed for a given invocation of a dynamic finder method. Here the QueryInfo.GetQueryElements method is called to extract the elements of the name as described above. The value returned by this method is QueryElement list option where:
type QueryElement =
| ElementQuery of string
| SubElementQuery of string * string
ElementQuery specifies the "Favorited" part in FindAllByFavoritedAlsoByUserWithScreen and the SubElementQuery belongs to the "ByUserWithScreen" part in FindAllByFavoritedAlsoByUserWithScreen .If the name of the invoked method corresponds is a supported name for a finder, the
GenerateCodeForBinder is called to generate the expression tree. The last argument of this method is a collection of the arguments provided for this invocation.
member this.GenerateCodeForBinder(elements, arguments : Expression array) =
let whereParameter = Expression.Parameter(typeof<JToken>, "c") in
let tmpVar = Expression.Parameter(typeof<JToken>, "tmp") in
let whereMethodInfo =
(typeof<System.Linq.Enumerable>).GetMethods()
|> Seq.filter (fun (m:MethodInfo) -> m.Name = "Where" && (m.GetParameters().Length = 2))
|> Seq.map (fun (m:MethodInfo) -> m.MakeGenericMethod(typeof<JToken>))
|> Seq.hd
let queryElementsConditions =
elements
|> Seq.zip arguments
|> Seq.map
(fun (argument,queryParameter) ->
this.GetPropertyExpressionForQueryArgument(queryParameter,argument,whereParameter,tmpVar)) in
let initialCondition = Expression.TypeIs(whereParameter,typeof<JObject>) in
let resultingExpression =
Expression.Block(
[tmpVar],
Expression.Call(
whereMethodInfo,
Expression.Property(
Expression.Convert(
this.Expression,this.LimitType),"array"),
Expression.Lambda(
Seq.fold
(fun s c -> Expression.And(s,c) :> Expression)
(initialCondition :> Expression)
queryElementsConditions,
whereParameter))) in
resultingExpression
The most important parts of this method is the definition of
queryElementsConditions and resultingExpression. The resulting expression specifies the invocation to the Where method The
queryElementsConditions take each argument extracted from the name of the method and tries to generate the necessary conditions for the value provided as an argument. In order to do this the GetPropertyExpressionForQueryArgument method is used:
member this.GetPropertyExpressionForQueryArgument(parameter:QueryElement,argument,cParam,tmpVar) : Expression =
match parameter with
| ElementQuery(propertyName) ->
this.CompareExpression(
this.GetJObjectPropertyExpression(cParam,propertyName) ,
argument)
| SubElementQuery(propertyName,subPropertyName) ->
Expression.And(
Expression.NotEqual(
Expression.TypeAs(
Expression.Assign(
tmpVar,
this.GetJObjectPropertyExpression(cParam,propertyName)),
typeof<JObject>),
Expression.Constant(null)),
this.CompareExpression(
this.GetJObjectPropertyExpression(
tmpVar,
subPropertyName),argument)) :> Expression
This method generates a different expression depending on the kind of query element that is requested.
Considerations for IronPython
As a curious note, in IronPython the
BindInvokeMember method is not called in the FSDynArrayWrapperMetaObject when a method is invoked. It seems that IronPython calls the BindGetMember method and then tries to apply the result of getting the method. So to make this object work with IronPython a implementation of the
BindGetMember method was created that returns a lambda expression tree with the generated Where invocation.
override this.BindGetMember(binder: GetMemberBinder) =
match QueryInfo.GetQueryElements(binder.Name) with
| Some( elements ) ->
let parameters =
List.mapi ( fun i _ ->
Expression.Parameter(
typeof<string>,
sprintf "p%d" i)) elements
(new DynamicMetaObject(
Expression.Lambda(
this.GenerateCodeForBinder(
elements,
parameters
|> List.map (fun p -> p :> Expression)
|> List.to_array ),
parameters),
binder.FallbackGetMember(this).Restrictions))
| None -> base.BindGetMember(binder)
Accessing using different names
The
QueryInfo.GetQueryElements method is used to allow the "FindAllBy..." and "find_all_by..." method names .
module QueryInfo = begin
...
let GetQueryElements(methodName:string) =
match methodName with
| str when str.StartsWith("FindAllBy") ->
Some(ExtractQueryElements(str.Substring("FindAllBy".Length),"AlsoBy","With"))
| str when str.StartsWith("find_all_by_") ->
Some(ExtractQueryElements(str.Substring("find_all_by_".Length),"_also_by_","_with_") )
| _ -> None
end
Code
Code for this post can be found here.
