Newspeak FFI
Newspeak provides a nice mechanism to call C code. This mechanism is described in
Newspeak Foreign Function Interface User Guide document. The
AlienDemo example provided with the Newspeak prototype has some nice small examples of the FFI.The experiment presented in this post was created using the Newspeak prototype from February 2009. Due to some limitations of this release, this code only works with the Windows version of the prototype.
libcurl
libcurl is a C library that provides client access to several networking protocols with a common interface. For this post I'm going to implement a wrapper for very small subset of the functionality provided by libcurl in order to perform simple HTTP/HTTPS GET and POST requests .
The simple.c example shows how to do a simple GET request.
int main(void)
{
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "curl.haxx.se");
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
return 0;
}
The LibCurlHelper class
A class named LibCurlHelper will be used to encapsulate calls to libcurl. As you will notice the interface of this class is pretty low level. For future posts I'll try to create a better interface using more Newspeak features.
class LibCurlHelper usingLib: platform = (
"This class wraps an implementation of the libcurl library"
|
Transcript = platform Transcript.
Alien = platform Alien.
UnsafeAlien = platform UnsafeAlien.
Callback = platform Callback.
CurlWriteCallback = platform CurlWriteCallbackNs1.
CurlDebugCallback = platform CurlDebugCallback.
ByteString = platform ByteString.
OrderedCollection = platform OrderedCollection.
...
public libcurlPath = ''.
public errorBuffer = ''.
protected CURL_OPT_URL = 10002.
protected CURLOPT_WRITEFUNCTION = 20011.
...
internalDebugCallback = nil.
internalWriteCallback = nil.
formPostData = nil.
private curlInstance = nil.
private aliensToRelease = nil.
CURLFORM_NOTHING = 0 .
CURLFORM_COPYNAME = 1 .
CURLFORM_PTRNAME = 2.
...
libcurl uses a lot of constaints prefixed with "CURL" this class contains definitions for some of them.
Initialization
The
initializeCurl method calls the curl_easy_init (as in the simple.c example shown above) and stores the returned pointer in a slot called curlInstance which will be used in further calls.
initializeCurl = (
|curl|
ensureLibrariesLoaded .
(Alien lookup: 'curl_easy_init' inLibrary: curlLibName )
primFFICallResult: (curl:: Alien new: 4).
curlInstance: curl.
)
The
ensureLibrariesLoaded and
curlLibName = (
^libcurlPath, 'libcurl.dll'
)
ensureLibrariesLoaded = (
Alien ensureLoaded: libcurlPath, 'libidn-11.dll'.
Alien ensureLoaded: libcurlPath, 'libeay32.dll'.
Alien ensureLoaded: libcurlPath, 'libssl32.dll'.
Alien ensureLoaded: curlLibName.
)
Setting the URL
In order to set the URL for the request we need to call the
curl_easy_setopt function with the CURL_OPT_URL with the URL string.The code looks like this:
url: url <String> = (
|result|
(Alien lookup: 'curl_easy_setopt' inLibrary: curlLibName )
primFFICallResult: (result:: Alien new:4)
withArguments: { curlInstance.
CURL_OPT_URL.
(addAlienToRelease: (url asAlien)) pointer. }.
^result.
)
The
addAlienToRelease: method was added to in order to keep track of resources allocated in the C heap that need to be manually released when not needed. The asAlien method of the String class creates a resource of this kind.The implementation of this method looks like this:
addAlienToRelease: anAlien = (
aliensToRelease isNil ifTrue: [ aliensToRelease:: OrderedCollection new. ].
aliensToRelease add: anAlien.
^anAlien.
)
Setting the write callback
Callback functions are used by libcurl to process the data coming from the network. The Newspeak FFI provides a nice way to add this kind of callbacks.
writeCallback: callback <Block>= (
|result|
internalWriteCallback:: Callback
block: callback
argsClass: CurlWriteCallback.
(Alien lookup: 'curl_easy_setopt' inLibrary: curlLibName )
primFFICallResult: (result:: Alien new: 4)
withArguments: { curlInstance.
CURLOPT_WRITEFUNCTION.
internalWriteCallback thunk. }.
^result.
)
The writeCallback: method sets the block in
callback as the libcurl write callback. In order to do this it creates an instance of the Callback class with the block and the arguments class. An instance of this class is used to create a function pointer which is passed to the curl_easy_setopt function.The "arguments class" is defined using the NS1 Newspeak syntax as follows:
Newsqueak1
'LangexplrExperiments'
CurlWriteCallbackNs1 = Alien (
"Class used to represent arguments of the LibCurl write function."
'as yet unclassified'
data = (
^Alien forPointer: (self unsignedLongAt: 1)
)
datasize = (
^(self unsignedLongAt: 5)
)
nmemb = (
^(self unsignedLongAt: 9)
)
writerData = (
^Alien forPointer: (self unsignedLongAt: 13)
)
) : (
'as yet unclassified'
dataSize = (
^16
))
An instance of this class is used to represent the arguments of a callback call. An example of the use of this function is presented below.
Performing the request
The
curl_easy_perform function is used to start the operation. The following code shows the call to this function:
performOperation = (
|r|
(Alien lookup: 'curl_easy_perform' inLibrary: curlLibName )
primFFICallResult: (r:: Alien new: 4)
withArguments: { curlInstance. }.
^r signedLongAt: 1.
)
Cleanup
Finally the following method is used to release the resources allocated by libcurl.
cleanup = (
(Alien lookup: 'curl_easy_cleanup' inLibrary: curlLibName )
primFFICallResult: nil
withArguments: { curlInstance } .
aliensToRelease do: [:anAlien | anAlien free ].
)
Example of using the library
As mentioned above, the
LibCurlHelper class provides a low level interface to libcurl, something needs to be created to encapsulate this functionality. The following method shows a method that preforms a simple GET request and returns the downloaded data as a string.
class HttpServiceClient usingLib: platform withCurlPath: curlLibraryPath = (
"This class is used to access services provided by the HTTP protocol"
|
LibCurlHelper = platform LibCurlHelper.
ByteString = platform ByteString.
platform = platform.
Transcript = platform Transcript .
private curlLibraryPath = curlLibraryPath .
|
)
(
simpleGet: url ^ = (
| curl data tmpBuffer bufferLength response|
curl:: (LibCurlHelper usingLib: platform).
curl libcurlPath: curlLibraryPath .
curl initializeCurl.
data:: ''.
curl:: createNewCurlInstance.
curl writeCallback:
[:args :result|
bufferLength:: ((args datasize) * (args nmemb)).
tmpBuffer:: ByteString new: bufferLength.
args data copyInto: tmpBuffer
from: 1 to: bufferLength
in: (args data) startingAt: 1.
data:: data,tmpBuffer.
result returnInteger: bufferLength.
].
curl url: url.
curl performOperation.
curl cleanup.
^data
)
)
Notice that here the callback function modifies a local variable every time the data arrives. Also notice that
args is an instance of CurlWriteCallbackNs1.Final words
The experiment of using libcurl from Newspeak was a nice way to learn about its foreign function interface. Having access to
libcurl access to useful things such as HTTPS requests.There's already a nice Squeak wrapper for libcurl called CurlPlugin .
Code for this post can be found here.
