Lisp

One of the bigger practical problems with unit testing is isolating the test coverage. Say, you want to test a piece of code from the middle (business) layer. Let's assume further the piece of code under consideration makes some calls to lower level code to retrieve some data. The problem of test coverage isolation is now that if you "simply" call your function, you are implicitly also testing the lower level code, which you shouldn't: if that lower level code gets modified in an incorrect way, you would suddenly see your middle level code fail although there was no change made to it. Let's explore ways to avoid the problems in Common Lisp.

There is a very good reason why you would also want to have such test dependencies to ensure your middle level code still works if the lower level code is extended or modified. But that is no longer unit testing: you are then doing so-called integration tests which are related, but still different beasts.

Now, I was facing exactly the typical dreaded situation: I extended an application right above the database access layer which had not seen much tests yet. And of course, I didn't want to go the long way (which I will eventually have to go anyway) and set up a test database with test data, write setup and tear-down code for the db etc. The typical suggestion (for the xUnit crowd) is to use mock objects which brings us finally on topic.
I was wondering if there are any frameworks for testing with mock objects in Lisp, but a quick search didn't turn up any results (please correct me if I've missed something). After giving the issue a little thought, it seemed quite clear why there aren't any: probably because it's easy enough to use home-grown solutions such as mine.

I'll use xlunit as the test framework, but that's not relevant. Let's look at some sample code we'll want to test:

(defun compare-data (data &connection)
  (let ((dbdata (retrieve-data-by-id (id data))))
    (when (equal (some-value data)
                (some-db-specific-value dbdata))
         t)))

The issue is with retrieve-data-by-id which is our interface to the lower level database access.
And note that we'll use some special functions on the results, too, even if they may just be accessors.
Let's assume the following test code:

(use-package :xlunit)

(defclass comp-data-tc (test-case)
  ((testdata :accessor testdata :initform (make-test-data))))

(def-test-method comp-data-test ((tc comp-data-tc))
  (let ((result (compare-data (testdata tc))))
    (assert-equal result t)))

Now the trouble is: given the code as it is now, the only way to succeed the test is to make sure that make-test-data returns an object whose values match values in the database you're going to use when compare-data get's called. You're ultimately tying your test code (especially the result of make-test-data) to a particular state of a particular database, which is clearly unfortunate. To overcome that problem, we'll use mock objects and mock functions. Let's define a mock-object mock-data and a mock-retrieve-data function, which will simply return a single default mock object.

(defclass mock-data ()
  ((id :accessor id :initarg :id :initform 0)
   (val :accessor some-db-specific-value :initarg :val :initform "foo-0"))))

(defun mock-retrieve-data (testcase)
  (format t "Establish mock for retrieve-data")
  (lambda (id)
      (format t "mock retrieve-data id:~A~%")
      (find-if #'(lambda (elem
			(when (equal (id elem) id)
			  elem))
                    (testdbdata testcase))))

Why that mock-retrieve-data returns a closure will become clear in a second, after we've answered the question how these entirely different named object and function can be of any help. The answer lies in CLs facility to assign different values (or better said) definitions to variables (or better said to function slots of symbols). What we'll do is to simply assign the function definition we've just created as the function to use when retrieve-data is going to be called. This happens in the setup code of the test case:

(defclass comp-data-tc (test-case)
  ((testdata :accessor testdata :initform (make-test-data))
   (testdbdata :accessor testdbdata)
   (func  :accessor old-retrieve-func)))

(defmethod set-up ((tc comp-data-tc))
  ; set up some test data
  (dotimes (number 9)
    (setf (testdbdata tc)
             (append 
                  (list 
                      (make-instance 'mock-data 
                            :id number 
                            :value (concatenate 'string "value-" number)))
                 (testdbdata tc))))
  ; establish our mock function
  (when (fboundp 'retrieve-data)
    (setf (old-retrieve-func tc)
             (fdefinition 'retrieve-data))))
  (setf (fdefinition 'retrieve-data)
           (mock-retrieve-data tc)))

(defmethod tear-down ((tc comp-data-tc))
   ; After the test has run, re-establish the old definition
   (when (old-retrieve-func tc)
      (setf (fdefinition 'retrieve-data)
           (old-retrieve-func tc))))

You can now see why mock-retrieve-data returns a closure: by this way, we can hand the data we establish for the test case down to the mock function without resorting to global variables.

Now, the accessor fdefinition comes in extremely handy here: we use it to assign a different function definition to the symbol retrieve-data which will then be called during the unit-test of compare-data.

..Establish mock for retrieve-data
mock retrieve-data id: 0
F
Time: 0.013


There was 1 failure: ...

There is also symbol-function which could be applied similarly and which might be used to tackle macros and special operators. However, the nice picture isn't as complete as one would like it: methods aren't covered, for instance. And it probably also won't work if the function to mock is used inside a macro. There are probably many more edge cases not covered by the simple approach outlined above. Perhaps lispers smarter than me have found easy solutions for these, too, in which case I would like to learn more about them.

In case you're interested in continuation-based web development with Common Lisp, it might be of interest to learn that Drew Campsie has voluntered to take over UCW development. Marco Baringer, initiator and maintainer of UncommonWeb (aka UCW) seemed somwhat relieved that finally somebody with more time on his hand steps up to take over.

That Marco hadn't had the time to keep pushing UCW is quite obvious from the mailing list archive. Lately Attila Lendvai seemed to me (an innocent external observer) to had pushed UCW foreward, however, as he Attila announced, he wants to go into a different direction than Drew.

I've been using ucw-dev during the last two years and the current situation really called for a change, in my opinion. UCWs main problem, as I observe it, is lacking documentation and clearly stated goals which way UCW will go. The automatically extracted documentation doesn't contain information how the bits fit together and all tutorials are a) incomplete, b) outdated as far as they are targetting ucw-dev, whereas development has mainly happened in ucw-ajax. However, it was not all clear that ucw-ajax really would become the main line of UCW.

I like UCWs component architecture a lot and will eagerly follow what is going to happen. With two prominent developers such as Marco and Attila more or less leaving the project, it will be interesting to see with how much activity development will happen from now on.

This post is mainly a reference post about a particular topic whose solution wasn't immediately obvious to me from the docs to CL-SQL. Using CL-SQL with (enable-sql-reader-syntax), I had written a routine that looks basically likes this:

(defun data-by-some-criteria (criteria &key (dbspec +db-spec+)  (dbtype +db-type+))
  (with-database (db dbspec :database-type dbtype :if-exists :old)
    (let (dbresult)
      (if criteria
	  (setq dbresult
		(select 'some-model 'other-model
			:where [and [= [some.criteria] criteria]
			            [= [some.foreign_id] [other.id]]]
			:order-by '([other.name] [some.foreign_id] 
				          [year] [some.name])
			:database db))
          (setq dbresult
		(select 'some-model 'other-model
			:where [and [null [some.criteria]]
			            [= [some.foreign_id] [other.id]]]
			:order-by '([other.name] [some.foreign_id] 
				          [year] [some.name])
			:database db))
      (when dbresult
	(loop for (some other) in dbresult
	      collect some)))))

This is ugly because the only difference between those two select statements is the check for the criteria, but I had no idea how to combine the two select statements into one, because it's not possible to embed lisp code (apart from symbols) into an sql-expression (i.e. the type of arguments for :where or :order etc.).

With the next requirement things would become far worse: The order-by statement needs to get more flexible so that it is possible to sort results by year first. Given the approach shown above this would result in at least four select statements, which is horrible. So, naturally I wanted a single select statement with programmatically obtained :where and :order-by sql expressions.

Step 1: It occured to me that it should be possible to have the arguments in a variable and simply refer to the variable. E.g., using a more simple example:

(let (where-arg)
  (if (exact-comp-needed)
      (setq where-arg '[= [column] someval])
      (setq where-arg '[like [column] someval]))
  (select 'model :where where-arg))

So I could now have my two different where-args and two different order-args and use a single select statement. Main problem solved.

Step 2: But for the :where arg in my original problem, only a small fraction of the sql-expression differs. So how do I avoid hard coding the entire value of where-arg? How can I combine some variable part of an sql-expression with some fixed parts? I.e, ultimately I want something like:

(let (comp-op where-arg)
  (if (exact-comp-needed)
      (setq comp-op '=)
      (setq comp-op 'like))
  (setq where-arg '[ <put comp-op here> [column1] someval])
  (select 'model :where where-arg))

But with CL-SQL modifying the reader, there seems to be no way to make <put comp-op here> work. I didn't knew how to get the usual variable evaluation into the sql-expression, or how to escape from CL-SQL's sql-reader-syntax to normal lisp evaluation.

Somewhere in the back of my head where was that itch that CL-SQL might offer some low-level access to sql expressions. And indeed it does. There are two useful functions, sql-expression and sql-operation. sql-operation "returns an SQL expression constructed from the supplied SQL operator or function operator and its arguments args" (from the cl-sql docs), and we can supply the operator and its arguments from lisp -- which is exactly what I want.

Now, the nice thing is that it's easy to mix partly handcrafted sql expressions with CL-SQL special sql syntax constructs that will be automatically handled by the reader (if you enable it only via enable-sql-reader-syntax, of course). I.e., for <put comp-op here> we can use sql-operation, but the rest stays essentially the same:

(let (where-arg)
  (if (exact-comp-needed)
      (setq where-arg (sql-operation '= [column1] someval))
      (setq where-arg (sql-operation 'like [column1] someval)))
  (select 'model 'other-model  :where where-arg))

Now, coming back to my original problem, based on this approach I can split out the common part of the :where and :order arguments and combine those with the varying parts as needed and hand them down to a single select statement. Problem solved.

This is just a minor rambling inspired by a recent thread on cll about the ups and downs of using Rails. What I find really overwhelming is the lack of proofs that other (typically Lisp-based) frameworks really have so much benefit over the established more traditional frameworks (e.g. Rails, Zope, plain PHP, ...). All I can see is starter documents, tutorials etc. that show how to program yet another blog or reddit clone in some particular framework. I miss fair comparisons and detailed discussions why and where exactly that one particular way of doing things has benefits. And especially with regard to those reddit clone prototypes: AFAICT, reddit was far beyond implementing some half-baked prototype (in Lisp) when they decided to switch (to Python). Yes, for demo purposes reinventing the wheel with some unround edges might be acceptable, but in reality only rolling wheels will be sold and this is where the challenge is.

I believe that while it's nice that frameworks help with implementing your 500 line application, they really need to show their value with large applications. Where is a discussion of a large on-line shopping system in, say, UCW and CL-SQL, with secure shopping over SSL, login handling, input validation, distribution over multiple servers etc.? Now, I'm not suggesting that UCW, Weblocks etc. can't be used to build such websites, but it would be nice to see a much more in-depth explanation/discussion/comparison of how to do it. That would also help increase the credibility of Lisp or at least, of lispers discussing (other) web frameworks.

In case you're interested in object-oriented programming in Lisp and happen to speak German, you'll be happy to learn that Pascal Costanza of ContextL and AspectL fame (apart of his postings to cll), has given a talk about CLOS, the Common Lisp Object System at the Hasso-Plattner Institut, Potsdam. And while we're at it, Pascal also has a very interesting blog post about the origins of the advice facility.

German:
Falls Dich, lieber Leser, interessiert, wie man objekt-orientiert in Lisp programmiert, solltest Du Dich darüber freuen, dass Pascal Costanza, bekannt durch ContextL und AspectL (abgesehen von seinen Postings in cll), einen Vortrag in Deutsch über CLOS, dem Common Lisp Object System, am Hasso Plattner Institut in Potsdam gehalten hat. Und weil wir gerade dabei sind, Pascal hat außerdem einen sehr interessanten Blogeintrag über die Geschichte von 'advice'.

Every now and then, I'm still blown away by the elegant power that the repl (read-eval-print-loop) of Common Lisp provides for everyday tasks. A recent example: I needed to fix a broken column definition in one of my Postgres databases. I'm not a DB pro, so I will just drop the column. But of course, we want to retain the old data, so here we go (using CL-SQL):

SOMEDB> (connect +db-spec+)
#
SOMEDB> (defvar *olddata* (select [id] [data] :from [sometable]))
*OLDDATA*
SOMEDB> (execute-command "ALTER TABLE sometable DROP COLUMN data;")
NIL
SOMEDB> (execute-command "ALTER TABLE sometable ADD COLUMN data varchar;")
NIL
SOMEDB> (loop for (id value) in *olddata*
                  do (update-records [sometable] 
                                 :attributes '(data) 
                                 :values (list value) 
                                 :where [= [id] id]))
NIL
SOMEDB> (disconnect)

That's it. First we open a connection, store away the olddata (which will be returned as a list of tuples) in a global variable, modify the table and finally restore the data. Now, what I find really nice about this is that I can operate on the data as if I had an intersection of psql, shell and a real programming language, which is pretty much the point of this blog post. I think that Ruby probably provides a similar environment with irb. And, hey, today I learned that XEmacs comes with an interface to postgres, so I might have been able to do it from within my favourite editor, too ...

Some time ago, I was looking at splitting some text with Elisp, Perl, Ruby and Common Lisp. Yesterday, when I again had to do quite the same thing, it occurred to me that the Common Lisp solution was unnecessary complex/long. I'm not a Perl guru, but I believe the following is probably hard to beat even with Perl:

CL-USER> (format t "~{<li>~A</li>~%~}" (cl-ppcre:split "\\|" "Kim Wilde|Transvision Vamp|Ideal|Siouxsie and the Banshees|Nena|Iggy Pop"))

<li>Kim Wilde</li>
<li>Transvision Vamp</li>
<li>Ideal</li>
<li>Siouxsie and the Banshees</li>
<li>Nena</li>
<li>Iggy Pop</li>
NIL

For the uninitiated, it's not the cl-ppcre library which is interesting here but the built-in iteration facilities of format. See the Hyperspec on the control-flow features of format for details. Now, I usually tend to avoid the mini languages that come with Common Lisp like the one of format or loop when writing real programs, but when using Lisp as a glorified shell they come in very handy.

Dear Lazyweb, I'm looking for a way to automate unit testing with the help of ASDF. I'm using XLUNIT at the moment, but this isn't really relevant. What I want to achieve is that on every compilation of some source file, it's corresponding test file will be loaded (and hence the tests it contains run). However, what I seek to avoid is simply adding the test files to the component definition; the test files should be kept separately. From what I gather from the ASDF documentation, this should be possible using :perform forms, but the very same docs leave me wondering how. What I found (looking at Jörg Höhles asdf file for iterate) is how to load and run a complete test-system, but this is not what I want to do. Ideally, I would like to have a single perform instruction looking something like this:

(defmethod asdf:perform ((op test-op) c)
  (when (asdf:glorified-find-component "my-component" c)
        (let ((component-test (find-component-test c)))
            (when component-test
                   (asdf:operate 'asdf:load-op 'component-test)))))

Here #'glorified-find-component needs to recursively follow the components parent and #'find-component-test should return a component c-test (with c expanded, obviously). Now, I guess I'm just clumsily reinventing the wheel and hence I'm wondering if somebody has already solved "the problem".

While I'm at it (it referring to testing), Stefil looks like a very interesting test environment in case you're developing your programs with Emacs and Slime. From there, I found a link to Phil Gregory's test framework comparison which I found much more enlightening than the ALUs list of test frameworks.

Recently, there was a discussion about "the rise of functional languages" over on Linux weekly news, in which one of the participant claimed that one of the major reasons why nobody uses functional languages in industrial settings would be the lack of explicit resource handling (where a resource is some supposedly "alien" object in the system, say a database handle or something like that). What he was referring to was the inability to run code on allocating/deallocating a piece of resource. Of course, some people pointed him to various solutions, in particular I recurred to the usual WITH-* style-macros in which one would nest the access to the data while at the same time hiding all what one would do on allocation/de-allocation. His reply went something along the lines that such objects may need to be long-lived (thus a WITH-macro is inappropriate) and that the only resort would be the garbage collector and that there simply is no way of running code at a guaranteed (de-allocation) time. I have to admit that I have no idea how I could code around that problem in Common Lisp (garbage collection even isn't a defined term in the ANSI specification of CL, and I'm very sure I haven't seen any mention of allocation/deallocation in it).

Now, some months later, there is a discussion in comp.lang.lisp on the topic of "portable finalizers" and Rainer Joswig pointed to this chapter in the Lisp machine manual which talks about explicit resource handling the lisp machine. From the excerpt, I can't judge whether resources are first-class CLOS objects and hence the functions to handle them are generic functions, but if so that would actually allow running code on deallocating a resource, of course with the price of having to handle allocation/deallocation manually. I really wonder if any of todays CL implementations offers the same or at least similar functionality?

Using Common Lisp, I want to split a list into two parts, in which the first resulting part has a fixed size (say 20 elements) and the rest, well, contains the rest of the list. This should also work when the list is actually shorter than the limit, which means I can't use subseq. The typically approach I would have chosen would involve the two traditional functions for doing something iteratively in CL, i.e. do, dotimes or dolist. However, this is ugly, and for the sake of learning something new, I had a look at how to solve the problem with loop.

Today, I was bitten by this problem: Illegal :ASCII character starting at byte position 16. I had some data I wanted to exchange with my Postgres DB but somehow, my Emacs consistently lost his connection to SBCL when I hit data with special chars (German umlauts encoded in ISO Latin 1). Investigating further, it became clear that SBCL had the problem without Slime, too. I was rather surprised then to see the solution in the thread linked to above: It was indeed a slime setting that fixed the problem. On a second look, I don't encounter the exact same problem, as I have trouble with encoded data in the database, not with the initial connect. But the solution suggested by Nikodemus Siivola helps me as well. There have been at last two other threads on the mailing list with regard to Unicode and Postgres, the last one is from April this year. Somebody from the slime camp could probably explain what that slime-setting does to SBCL that it affects SBCL connection to Postgres via CL-SQL. Perhaps I'll go and have a look. Ah, the joys of open source.

Update:I was under a wrong impression. I previously had set SB-IMPL::*default-external-format* to :LATIN-1, a suggestion from the same thread. The slime setting only handles the communication issue with Emacs, but setting the variable on the sbcl side fixes the real problem.

Which Programming Language are You?

Moving back in time is easy when the tools you're using are a versioning control system like CVS or RCS [1], (X)Emacs, it's versioning control interface VC, a Common Lisp system such as SBCL and Slime, the superior lisp interaction mode for Emacs. Yesterday, I was debugging a part of my current hobby project which involves a web interface made with Uncommon Web (UCW). One bit of that web interface that I hadn't paid particular attention to in the last round of modifications had stopped working. But the source that dealt with that particular bit seemed quite innocent to me.

I then looked through the log of modifications I did to the file (which is under version control via RCS) by hitting "C-x v l" (equivalent to "M-x vc-print-log"). Turned out that I did some modifications between versions 1.5 and 1.6 to the part under consideration. Hitting "C-x v ~" (vc-version-other-window) and entering numbers 1.5 (and afterwards 1.6) re-generated the old versions, XEmacs directly visiting these corresponding buffers. This feature alone is worth noting: When extracting the older versions, the current version won't be overwrittten, the older versions will be put into the filesystem with the version number added as a suffix, e.g. foo.lisp.~1.5~.

Now, to see what was happening in between those two versions, I first looked at the differences between the two files. "M-x ediff" and selecting the two buffers, it was easy to spot the part of the code I changed. Next, have a look at the effects caused by the changes: For that I did a "C-c C-k" (slime-compile-and-load-file) on the old versions respectively, so I could then see the old behaviour in real life -- i.e., I could directly use the web app in that older version in my Firefox without any further ado. Of course, this also has to do with the fact that there weren't many interactions between the part of the app in question and other parts of the system, in particular I needn't to check out older versions of other files for the compilation to succeed. Actually, loading that old version certainly broke other functionality of the app, but that wasn't the issue: I only wanted to figure out how the old version worked wrt. to that particular part that wasn't working in my current state, hence a fully working old version wasn't what I was looking for.

Now, it did require a little more close attention to the changes I made to discover the mistake. But I think it's amazing in how few steps it's possible to regenerate an old behaviour. All the while, my current state of affairs remained largely untouched, I didn't need to create any copies or move files around or do a larger recompilation. I think that's a highly effective way of discovering the introduction of a bug.

Update: Kent Pitman, who's giving cll another of his (lately very rare) visits, pointed to one of his older papers (Accelerating Hindsight), which I'm surely have seen before but totally forgotten, which talks about how good Common Lisp is for rapid prototyping. What I've shown above is actually just an example of two points in this paper: dynamic redefinition and editor integration. Go figure.

Footnotes:
[1] Yeah, I know, using rcs or cvs and not bazaar, mercurial or git makes me look like an oldtimer. That said, I do use subversion in my day job, too. I stick with rcs and cvs for smaller (read: one person involved) projects mainly due to one reason: There is no support for the newer kids on the block in the VC shipping with XEmacs. Yes, I know that the VC shipping with FSF Emacs has support for subversion (at least), but unfortunately it's not compatible.

Reading Bill Clementson's blog pointed me to Edi Weitz new STARTER-PACK Lisp package, which tries to overcome one of Common Lisp perceived entry level problems for newcomers, when compared to e.g. Python. Python is known to come with "batteries included" which actually means that you'll have lots of libraries for everyday problems installed out of the box. For Common Lisp, not much has been accomplished in this area, as most people still seem to be debating on comp.lang.lisp about standarization. Edi's starter pack instead does the right thing: It provides a simple selection of common packages, some of which could be called to have achieved de-facto standard level (e.g. his own CL-PPCRE package). Unfortunately, it's for Windows and Lispworks only, which Bill ported to Mac/LW.
Being a Debian user since many years (should be roughly ten years now), I think that it would be fairly simple to achieve something similar (but not at all comparable) to STARTER-PACK. There are already a lot of Common Lisp packages in Debian, what's missing, however, is a CL-TASK package. Such a package could do what all other task-* packages do in Debian: Have dependencies on the most common packages. This would provide at least a starting point for the newbies with CL in Debian. I think, I'm going to discuss this with Peter van Eynde via mail and also on cll.

It has been claimed that Common Lisp would provide better error handling than other languages, where "better" equates equally to "provide superior error handling facilities and to "clear error messages". As a mere application programmer, I strongly believe in that this should carry over to libraries, but unfortunately lately I get the impression that this tradition isn't given enough respect by some of the more younger kids on the block.

As my pet peeve example of late, consider UCW: it's a prime example of how to assemble a lot of functionality by relying on other libraries. Unfortunately, standing on giants shoulders can be become quite a task if the giants start shaking. To me, UCW gives really horrible error messages. Most of the time, you'll get no idea where to look for the error.

As a simple example, when UCW encounters an error while parsing some tal file, you just might get the error message "NIL is not a character", which won't give you any clue what caused the problem. Sometimes, you won't get an error message at all, but simply get no result at all. As an example, I took one tal file from some test project, modified it a little and tested it on my new application. I didn't get any output from the variables referenced. After a lot of time spend debugging the lisp-side of the code, I finally saw the light: I forgot to change the lisp package referred to in the tal-file header, hence UCW tried (unsuccessfully) to access variables that didn't even exist. A warning (in UCWs log) would have been very helpful.

Another problem might be related to the code walking UCW does: sometimes you may get an error about unbound variables while trying to render some component. With UCW this problem occurs more often (and I think this might be due to the code walking), as sometimes a (with-slots (foo) component &body) is not sufficient to avoid the accessor, i.e. (foo component). Well, that kind of error isn't uncommon: it's easy to mistype a variable name or to forget (or -- as in UCWs case -- to not knowing that you should) use the syntax for a variable when instead you should have used a CLOS accessor. So usually, you'll take a quick glance at the error message or the backtrace and then fix the error in the erroneous piece of code. Not so with UCW: usually it's near to impossible to find the damned piece of code because the backtrace won't show you even the name of the component. The name of the variable typically will not be of much help either, if you happen to move state information through several components, (re-)using the same slot names all over the place.

So, to sum up, what I believe is that UCW falls short on is coping with the errors its building blocks may generate. The result is that while the resulting code for some application may be terse and even pretty [1], it may take a lot of time to get it right because of the amount of time wasted debugging unclear error messages. Please, library designers: support your local errors.

Footnotes:
[1] I don't think that the code for an UCW application is pretty, but that's another story.